CN111209429B - Unsupervised model training method and unsupervised model training device for measuring coverage of voice database - Google Patents

Abstract

The present disclosure relates to an unsupervised model training method for measuring speech database coverage, the method comprising: acquiring training data, wherein the training data is voice; determining one or more evaluation factors for voice database coverage; dividing the evaluation factors into adjustable factors or non-adjustable factors based on whether the training data can be controlled by parameter adjustment; determining a clustering algorithm corresponding to each divided evaluation factor; classifying the training data through a clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses; and training an evaluation model according to the plurality of subclasses of each evaluation factor. The method can set different voice databases corresponding to the evaluation element measurement according to the needs of users, purposefully extract different characteristics and select proper algorithms by distinguishing the evaluation elements, and meanwhile, model training can be carried out by using unsupervised data, so that the cost introduced by data annotation is reduced.

Description

Unsupervised model training method and device for measuring coverage of speech database

technical field

The present disclosure relates to the field of speech signal processing, and in particular, to an unsupervised model training method and apparatus, electronic device and computer-readable storage medium for measuring the coverage of a speech database.

Background technique

The coverage of the voice database is an important indicator to measure the quality of the voice database, which refers to the coverage of the voice database for evaluation factors. For example: the speaker's gender, language, voice content and other factors. For example, when training a speech recognition system, it is necessary to collect the speech of a large number of speakers for training. At this time, the better the coverage of the selected speech database, the more extensive the speech space can be included, which can effectively reduce the number of samples. influence in the spatial distribution.

The traditional acquisition of the coverage of the voice database relies on the expert experience in the design stage of the voice database. When making the acquisition plan, the voice in the voice database is distributed as comprehensively as possible on various evaluation factors. However, for the database that has been collected, indirect feedback can be obtained according to the recognition rate and other indicators only after the speech signal is processed and modeled. In the process of training the speech data evaluation model, it is difficult to construct an accurate evaluation model due to the incomplete division of the training data and the lack of manually annotated sample data.

SUMMARY OF THE INVENTION

In order to overcome the problems existing in the related art, the present disclosure provides an unsupervised model training method and apparatus, an electronic device and a computer-readable storage medium for measuring the coverage of a speech database.

According to a first aspect of the embodiments of the present disclosure, there is provided an unsupervised model training method for measuring the coverage of a speech database, the method comprising: acquiring training data, the training data is speech; determining one or more evaluations of the coverage of the speech database factors; based on whether the training data corresponds to the evaluation factors that can be controlled by parameter adjustment, divide the evaluation factors into adjustable factors or non-adjustable factors; determine the clustering algorithm corresponding to each evaluation factor after division; through the clustering algorithm corresponding to each evaluation factor The class algorithm separately classifies the training data to obtain multiple subclasses; according to the multiple subclasses of each evaluation factor, the evaluation model is trained.

In one embodiment, determining the clustering algorithm corresponding to each evaluation factor after the division includes: if the evaluation factor is an unadjustable factor, determining that the corresponding clustering algorithm is a distance-based clustering algorithm; if the evaluation factor is If the adjustable factor is set, the corresponding clustering algorithm is determined to be an adaptive training algorithm.

In one embodiment, the training data is classified by a clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses, including: if the evaluation factor is an unadjustable factor, extracting a feature vector of the training data; according to the feature vector, using A distance-based clustering algorithm that divides the training data into multiple subclasses.

In one embodiment, the distance-based clustering algorithm is a K-means clustering algorithm.

In one embodiment, the training data is classified by a clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses, including: if the evaluation factor is an adjustable factor, extracting a feature vector of the training data; Gaussian mixture model, labeled training data; according to the labeled training data, the training data is divided into multiple subclasses.

In one embodiment, training the Gaussian mixture model and labeling the training data through the feature vector includes: training the Gaussian mixture model through the feature vector; determining control parameters according to the evaluation factors, and the control parameters can be adjusted to control the training data; traversing all the control parameters. Take the value to transform the training data; obtain the parameter value when the eigenvector of the transformed training data maximizes the likelihood of the Gaussian mixture model; accumulate the likelihood according to the parameter value; transform the training data according to the parameter value to obtain a new training data, and retrain until the stopping condition is reached; the parameter value corresponding to each training data when the likelihood of the Gaussian mixture model is maximized is used as the label value of the training data.

In one embodiment, the stopping condition includes: the number of iterations reaches a preset threshold, or the rate of change between the accumulated likelihood and the accumulated likelihood in the previous iteration is less than a preset threshold.

In one embodiment, training an evaluation model according to multiple subclasses of each evaluation factor includes: training one or more evaluation models on each subclass data separately, or training a single evaluation model on a plurality of subclass data as a whole.

In one embodiment, the evaluation factors for the coverage of the speech database include one or more of the following: the speaker's gender, the speaker's age, the speaker's accent, speech rate, pitch, language, collection device, collection environment, and pronunciation factors or content topics.

According to a second aspect of the embodiments of the present disclosure, there is provided a method for measuring the coverage of a speech database, the method comprising: using the unsupervised model training method for measuring the coverage of a speech database as in the first aspect, obtaining the The evaluation model obtains the voice database to be evaluated, wherein the voice database includes at least one voice; each voice in the voice database is detected by the evaluation model of the evaluation factor, and the single-factor information entropy corresponding to the evaluation factor in the voice database is obtained ; According to the single factor information entropy, determine the coverage of the voice database.

According to a third aspect of the embodiments of the present disclosure, there is provided an unsupervised model training device for measuring the coverage of a speech database, the device comprising: a data acquisition unit for acquiring training data, where the training data is speech; an evaluation factor determination unit, One or more evaluation factors used to determine the coverage of the voice database; a division unit, used to divide the evaluation factors into adjustable factors or non-adjustable factors based on whether the training data corresponds to whether the evaluation factors can be controlled by parameter adjustment; the algorithm determination unit, It is used to determine the clustering algorithm corresponding to each evaluation factor after division; the classification unit is used to classify the training data through the clustering algorithm corresponding to each evaluation factor to obtain multiple subclasses; the model training unit is used to classify the training data according to each evaluation factor. Multiple subclasses of each evaluation factor to train the evaluation model.

In one embodiment, the algorithm determining unit is further configured to: when the evaluation factor is an unadjustable factor, determine that its corresponding clustering algorithm is a distance-based clustering algorithm; when the evaluation factor is an adjustable factor, determine its corresponding clustering algorithm. The clustering algorithm is an adaptive training algorithm.

In one embodiment, the classification unit is further configured to: extract a feature vector of the training data when the evaluation factor is an unadjustable factor; and divide the training data into multiple subclasses by using a distance-based clustering algorithm according to the feature vector.

In one embodiment, the distance-based clustering algorithm is a K-means clustering algorithm.

In one embodiment, the classification unit is further used to: extract the feature vector of the training data when the evaluation factor is an adjustable factor; train the Gaussian mixture model through the feature vector, and label the training data; into multiple subcategories.

In one embodiment, training the Gaussian mixture model and labeling the training data through the feature vector includes: training the Gaussian mixture model through the feature vector; determining control parameters according to the evaluation factors, and the control parameters can be adjusted to control the training data; traversing all the control parameters. Take the value to transform the training data; obtain the parameter value when the eigenvector of the transformed training data maximizes the likelihood of the Gaussian mixture model; accumulate the likelihood according to the parameter value; transform the training data according to the parameter value to obtain a new training data, and retrain until the stopping condition is reached; the parameter value corresponding to each training data when the likelihood of the Gaussian mixture model is maximized is used as the label value of the training data.

In one embodiment, the stopping condition includes: the number of iterations reaches a preset threshold, or the rate of change between the accumulated likelihood and the accumulated likelihood in the previous iteration is less than a preset threshold.

In one embodiment, the model training unit is further configured to: train one or more evaluation models on each sub-class data separately, or train a single evaluation model on a plurality of sub-class data as a whole.

In one embodiment, the evaluation factors for the coverage of the speech database include one or more of the following: the speaker's gender, the speaker's age, the speaker's accent, speech rate, pitch, language, collection device, collection environment, and pronunciation factors or content topics.

According to a fourth aspect of the embodiments of the present disclosure, there is provided an apparatus for measuring the coverage of a speech database. The apparatus includes an evaluation model obtaining unit for using the unsupervised model training method for measuring the coverage of a speech database according to the first aspect. , obtain the evaluation model of each evaluation factor, the voice database acquisition unit is used to obtain the voice database to be evaluated, wherein, the voice database includes at least one voice; the detection unit is used to pass the evaluation model of the evaluation factor to the voice database. Each voice is detected to obtain the single-factor information entropy of the voice database corresponding to the evaluation factors; the evaluation unit is used to determine the coverage of the voice database according to the single-factor information entropy.

According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, comprising: a memory for storing instructions; and a processor for invoking the instructions stored in the memory to execute the non-independent method for measuring the coverage of the voice database of the first aspect. Supervised model training methods.

According to a sixth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing instructions, and when the instructions are executed by a processor, the unsupervised model training method for measuring the coverage of a speech database of the first aspect is executed.

The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: the present disclosure proposes an unsupervised model training method for measuring the coverage of a speech database. First, different evaluation elements can be set according to user needs to measure the corresponding database, For example, when doing accent recognition and classification, an evaluation model can be built according to the user's needs by setting evaluation factors such as accent, pitch, speed of speech, etc., so that the speech database evaluation model can be more targeted in specific application scenarios; secondly, by distinguishing different evaluation factors, It is helpful to extract corresponding features and select appropriate algorithms in subsequent processing, and further build a more appropriate and accurate classification model based on the evaluation factors, which is helpful to accurately determine the coverage of the speech database in the future. Supervised data, that is, unlabeled data, is used for model training, so as to achieve quantitative evaluation of the coverage of the speech database and reduce the cost introduced by data labeling.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.

1 is a schematic flowchart of an unsupervised model training method for measuring the coverage of a speech database according to an exemplary embodiment;

2 is a schematic flowchart of another unsupervised model training method for measuring the coverage of a speech database according to an exemplary embodiment;

3 is a schematic flowchart of a method for measuring the coverage of a speech database according to an exemplary embodiment;

4 is a schematic block diagram of an unsupervised model training apparatus for measuring the coverage of a speech database according to an exemplary embodiment;

Fig. 5 is a schematic block diagram of an apparatus according to an exemplary embodiment.

Fig. 6 is a schematic block diagram of an electronic device according to an exemplary embodiment.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with some aspects of the invention as recited in the appended claims.

At present, the commonly used voice database coverage evaluation system relies on expert annotation data, and the division of training data is not comprehensive and lacks diversity. In some related technologies, relying on supervised data, that is, data with annotations, it is necessary to divide the data into several subclasses according to the annotations, and then train the model according to the labels of each subclass. These supervised data need to be manually labeled to obtain, which often requires a lot of manpower, material resources and time.

In order to solve the above problems, the present disclosure provides an unsupervised model training method 10 for measuring the coverage of a speech database. Referring to FIG. 1 , the method includes steps S11 to S15, which are described in detail below:

In step S11, training data is acquired, and the training data is speech.

Among them, the training data can come from one or more voice databases, which cover a wide range of voices and have multiple classification possibilities to ensure the diversity of data in model training.

Step S12, determining one or more evaluation factors for the coverage of the speech database.

In an embodiment of the present disclosure, the evaluation factors for the coverage of the speech database include one or more of the following: the speaker's gender, the speaker's age, the speaker's accent, speech rate, pitch, language, pronunciation factor, or content theme . In the evaluation process, users can determine the evaluation factors that need to be involved in the speech database according to the usage requirements of the speech database. For example, when doing accent recognition and classification, they can set up evaluation factors such as accent, pitch, and speech speed to build an evaluation model according to the user's needs. Through the steps of the above embodiments, the user can independently set the evaluation factors, so that the speech database evaluation model is more targeted in specific application scenarios, and the obtained evaluation results are more in line with the user's needs.

Step S13 , based on whether the training data corresponds to the evaluation factor that can be controlled by parameter adjustment, divide the evaluation factor into adjustable factors or non-adjustable factors.

Specifically, for a variable factor to be evaluated, first determine whether the speech corresponding to the evaluation factor can be adjusted, that is, there is a certain transformation method, so that it can be transformed from one value to another. If there is a transformation method, the evaluation factor is considered to be an adjustable factor. For example, when the evaluation factor is the speech rate, the time domain scaling can be used to make the speech rate faster or slower; for another example, the pitch can be reduced by frequency domain scaling. or increase; if there is no transformation method, the evaluation factor is considered to be an irreversible factor. Generally speaking, according to the current technical means, there is no simple transformation method for the irreducible factor to be determined by a value. Convert to another value, such as accent, language, etc. By distinguishing whether an evaluation element can be adjusted, it is helpful for developers to further select an appropriate processing method for the evaluation element, extract different features for different evaluation elements, and formulate a processing algorithm that is more in line with the element. In the subsequent construction of the overall evaluation model, the distinguished elements can make the model architecture clearer and further improve the accuracy of the evaluation model.

Step S14, determining a clustering algorithm corresponding to each of the divided evaluation factors.

In an embodiment of the present disclosure, if the evaluation factor is an unadjustable factor, the corresponding clustering algorithm is determined to be a distance-based clustering algorithm; if the evaluation factor is an adjustable factor, the corresponding clustering algorithm is determined to be Adaptive training algorithm. When there is no transformation method for the evaluation factor corresponding to the voice, the category of the factor is relatively clear. For example, the gender of the speaker includes two types of men and women, and the language category of the voice is relatively fixed. In this case, the distance-based clustering algorithm is more intuitive and convenient, easy to implement and has a high accuracy; however, when there are transformation methods for the evaluation factors corresponding to speech, the category information cannot be directly reflected in the feature vector, and only the measured features can be used. In this case, selecting the adaptive training algorithm can effectively set the characteristic parameters related to the evaluation factors, and classify the evaluation factors more accurately.

In step S15, the training data are classified respectively through the clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses.

In an embodiment of the present disclosure, the training data is classified by a clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses, including: if the evaluation factor is an irreversible factor, extracting a feature vector of the training data; A vector that uses a distance-based clustering algorithm to divide the training data into multiple subclasses.

Specifically, if the evaluation factor is an irreversible factor, a vector representation of the relevant features of the training data is extracted. For example, for factors such as accent, language, speaker, etc., i-vector vector, x-vector vector, etc. can be extracted; for factors such as environment, equipment, phoneme content, etc., MFCC (Mel Frequency Cepstral Coefficient) feature vector can be extracted, etc. According to the extracted features, the data is divided into multiple sub-categories. When extracting features, selecting feature vectors that can effectively distinguish different speeches under the evaluation factor helps to improve the accuracy of clustering.

In an embodiment of the present disclosure, the distance-based clustering algorithm is a K-means clustering algorithm. The K-means algorithm is easy to implement, and is very efficient and practical for computational clustering problems.

In an embodiment of the present disclosure, classifying the training data through a clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses, further comprising: if the evaluation factor is an adjustable factor, extracting a feature vector of the training data; Feature vector, train a Gaussian mixture model, label the training data; according to the labeled training data, divide the training data into multiple subclasses. Since the Gaussian mixture model is a probabilistic model, it assumes that all samples are generated from a mixture of a finite number of Gaussian distributions with unknown parameters. Any density function can be approximated as a linear combination of Gaussian probability density functions. Speech features usually have a smooth probability density function, so a limited number of Gaussian functions can form a smooth approximation to the density function of speech features, and can effectively distinguish speech for adjustable factors.

In an embodiment of the present disclosure, training a Gaussian mixture model through feature vectors and labeling training data includes: training a Gaussian mixture model through feature vectors; determining control parameters according to evaluation factors, and the control parameters can be adjusted to control the training data; traversal control All values of the parameters, transform the training data; obtain the parameter value when the eigenvector of the transformed training data maximizes the likelihood of the Gaussian mixture model; accumulate the likelihood according to the parameter value; transform the training data according to the parameter value, get New training data, re-train until the stopping condition is reached; the parameter value corresponding to each training data that maximizes the likelihood of the Gaussian mixture model is used as the labeling value of the training data.

In an embodiment of the present disclosure, the stopping condition includes: the number of iterations reaches a preset threshold, or a rate of change between the accumulated likelihood and the accumulated likelihood in the previous iteration is less than a preset threshold.

Specifically, MFCC feature vectors of all data can be collected, and a GMM model (Gaussian Mixed Model, Gaussian Mixed Model) can be trained;

，然后对所有数据逐一进行如下操作：Let likelihood accumulator

, and then do the following for all data one by one:

，得到

：例如对语速变换，

指时域缩放因子；对音调，

指频域缩放因子；The data is transformed, and the transformation control parameters are

,get

: For example, for speech rate change,

refers to the time domain scaling factor; for pitch,

refers to the frequency domain scaling factor;

值离散化，遍历所有取值，使得

的MFCC特征对GMM模型

的似然度最大化：right

Value discretization, traverse all values, so that

The MFCC features for the GMM model

to maximize the likelihood of:

的取值

：When the record reaches the maximum value

value of

:

累计似然度：By value

Cumulative Likelihood:

累对数据进行变换，得到新的数据：By value

Transform the data to get new data:

与上一次迭代变化值小于

，或迭代次数达到8次；Continue to collect MFCC eigenvectors of all data, train the GMM model to iterate until a stop condition is reached, in the example we set the stop condition to the likelihood accumulator

Change value from previous iteration is less than

, or the number of iterations reaches 8;

的取值

，将训练数据分成M个子类。According to the maximum value of each record recorded

value of

, divide the training data into M subclasses.

For speech classification under adjustable factors, since the feature vector itself cannot well reflect the adjustability of speech, the method of directly using vector distance to measure is not accurate for the clustering of adjustable factors. The feature vector is described by the likelihood of the Gaussian mixture model, and more speech feature information can be obtained in the process of feature transformation, which can have a more accurate and effective description effect on the distribution of speech data. At the same time, selecting the appropriate stopping condition helps to improve the speed of the algorithm while ensuring the accuracy of the algorithm.

In step S16, an evaluation model is trained according to the multiple sub-categories of each evaluation factor.

In an embodiment of the present disclosure, one or more evaluation models are separately trained for each subclass of data, or one evaluation model is trained as a whole for multiple subclasses of data. For multiple sub-categories of each evaluation factor, the corresponding evaluation model can be trained as required, so as to facilitate the subsequent determination of the information entropy of each speech in the speech database relative to the evaluation factor, so as to further determine the coverage of the speech database.

The process of an embodiment of the present disclosure is shown in FIG. 2 . First, the evaluation factors for the coverage of the voice database are determined; the evaluation factors are divided based on whether there is a transformation mode corresponding to the evaluation factors in the voice; if there is no transformation mode, the feature vector is directly extracted , use the k-means clustering algorithm for clustering, obtain multiple subclasses under the evaluation factor, and construct the model of the evaluation factor through the obtained subclasses; if there is a transformation method, extract the MFCC vector, build a GMM model, and pass the parameter transformation. The method determines the parameter value when the likelihood is the largest, transforms the vector, enters the GMM model to continue to iterate, and labels the data with the parameter value when the likelihood is the largest, so as to obtain multiple subclasses, and construct the subclass through the obtained subclass. A model for evaluating factors. In the above embodiment, for different evaluation elements, different feature vectors are extracted and corresponding clustering algorithms are selected, so that the evaluation model is more targeted; at the same time, the algorithm uses unsupervised data for model training throughout the process, which reduces the number of data. The cost introduced by the callout.

The present disclosure also provides a method 20 for measuring the coverage of a voice database. As shown in FIG. 3 , the method 20 for measuring the coverage of a voice database includes steps S21 to S24, and the details are as follows:

Step S21: Obtain an evaluation model for each evaluation factor by using the unsupervised model training method 10 for measuring the coverage of a speech database in any of the foregoing embodiments.

Step S22: Acquire a voice database to be evaluated, wherein the voice database includes at least one voice.

Step S23: Detecting each voice in the voice database through the evaluation model of the evaluation factors, and obtaining the single-factor information entropy corresponding to the evaluation factors in the voice database.

In the embodiment of the present disclosure, each voice in the voice database is detected separately according to the evaluation factors involved in determining the voice database. An evaluation model can detect one or more evaluation factors. Through the classification and detection of the evaluation model, the single-factor information entropy of each evaluation factor involved in the speech in the speech database is determined, and the possibility that the speech database is involved in each evaluation factor is obtained, which is convenient to determine whether the speech database involves the evaluation factor that needs to be involved, and the existence of the evaluation factors. probability. The more sub-category factors involved in each voice, the more scattered, the higher the entropy value of the single-factor information entropy obtained. On the contrary, the more concentrated the sub-category factors involved in each voice, the lower the obtained single-factor information entropy entropy value.

In one embodiment, based on the evaluation factors, the evaluation model is used to classify and detect each voice, and the subclass conditional probability corresponding to each voice in the voice database and the multiple subclass factors in the evaluation factor is obtained; based on the subclass conditional probability, obtain The single-factor information entropy of the speech database corresponding to the evaluation factors.

Specifically, according to the evaluation factors that need to be involved, the evaluation model will classify and detect each speech in the speech database, and determine the probability of each speech corresponding to each evaluation factor that needs to be involved. According to the detection, the conditional probability of each speech in the speech database corresponding to each sub-category factor under the current evaluation factor can be obtained, which is convenient to clarify the occurrence probability of each speech under the condition that each sub-type factor is involved in each evaluation factor. The subclass conditional probabilities of the speech under each subclass factor are integrated to obtain the single-factor information entropy of the speech database under this evaluation factor.

In one embodiment, for the current evaluation factor, by detecting each voice, the subclass conditional probabilities of each voice corresponding to each subclass factor under the current evaluation factor can be obtained, and they can be aggregated and averaged to obtain a voice database. Under the current evaluation factor, the subclass average conditional probability corresponding to each subclass factor is obtained, and then the single factor information entropy of the speech database under the current evaluation factor is obtained according to the subclass average conditional probability of the speech database under each subclass factor.

用于表示语音数据库，K表示语音数据库中语音的条数，

至

代表语音数据库中的每一条语音。

代表当前评价因素下中的各子类因素，M为子类因素的个数。通过评价模型，采用如下公式获取每条语音在各子类因素下的子类条件概率：

，

，

。K表示语音数据库中语音的条数，M为子类因素的个数。进而采用下述公式进行整合，得到语音数据库在各子类因素下的子类平均条件概率：

，

。从而根据下述公式得到语音数据库涉及当前评价因素的单因素信息熵：

，log为自然对数。In an implementation scenario,

Used to represent the voice database, K represents the number of voices in the voice database,

to

Represents each speech in the speech database.

represents each sub-category factor under the current evaluation factor, and M is the number of sub-category factors. Through the evaluation model, the following formula is used to obtain the subclass conditional probability of each speech under each subclass factor:

,

,

. K represents the number of voices in the voice database, and M is the number of subclass factors. Then, the following formula is used for integration to obtain the subclass average conditional probability of the speech database under each subclass factor:

,

. Thus, the single-factor information entropy of the speech database involving the current evaluation factor is obtained according to the following formula:

, log is the natural logarithm.

Step S24: Determine the coverage of the speech database according to the single-factor information entropy.

Specifically, according to the information entropy of each single factor obtained under each evaluation factor in the speech database, the number of classification factors and entropy values that need to be involved in each speech in the speech database can be quickly clarified. Judging the coverage of all voices, and then evaluating whether the voices in the voice database meet the requirements, so as to determine whether the quality of the voice database is qualified. The result of model training or other subsequent processing based on the speech database. By using the information entropy to judge the classification factors involved in the speech database and then to evaluate the quality of the speech database, it is beneficial to quantify the uncertain factors, uniformly measure the standards of each classification factor involved in each speech, and make the abstract judgment information concrete. In turn, it is helpful to obtain the quality of the speech database to be evaluated directly and quickly.

Based on the same inventive concept, FIG. 4 shows an unsupervised model training device 100 for measuring the coverage of a speech database, including: a data acquisition unit 110 for acquiring training data, where the training data is speech; an evaluation factor determining unit 120, for determining one or more evaluation factors of the coverage of the voice database; dividing unit 130, for dividing the evaluation factors into adjustable factors or non-adjustable factors based on whether the training data corresponds to whether the evaluation factors can be controlled by parameter adjustment; The determination unit 140 is used to determine the clustering algorithm corresponding to each evaluation factor after division; the classification unit 150 is used to classify the training data respectively through the clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses; the model training device 160, for training an evaluation model according to the multiple subclasses of each evaluation factor.

In one embodiment, the algorithm determining unit 140 is further configured to: when the evaluation factor is an unadjustable factor, determine that its corresponding clustering algorithm is a distance-based clustering algorithm; when the evaluation factor is an adjustable factor, determine its corresponding clustering algorithm. The clustering algorithm is an adaptive training algorithm.

In one embodiment, the classification unit 150 is further configured to: extract a feature vector of the training data when the evaluation factor is an unadjustable factor; and divide the training data into multiple subclasses by using a distance-based clustering algorithm according to the feature vector.

In one embodiment, the distance-based clustering algorithm is a K-means clustering algorithm.

In one embodiment, the classification unit 150 is further configured to: extract a feature vector of the training data when the evaluation factor is an adjustable factor; train a Gaussian mixture model through the feature vector, and label the training data; Data is divided into multiple subcategories.

In one embodiment, training the Gaussian mixture model and labeling the training data through the feature vector includes: training the Gaussian mixture model through the feature vector; determining control parameters according to the evaluation factors, and the control parameters can be adjusted to control the training data; traversing all the control parameters. Take the value to transform the training data; obtain the parameter value when the eigenvector of the transformed training data maximizes the likelihood of the Gaussian mixture model; accumulate the likelihood according to the parameter value; transform the training data according to the parameter value to obtain a new training data, and retrain until the stopping condition is reached; the parameter value corresponding to each training data when the likelihood of the Gaussian mixture model is maximized is used as the label value of the training data.

In one embodiment, the stopping condition includes: the number of iterations reaches a preset threshold, or the rate of change between the accumulated likelihood and the accumulated likelihood in the previous iteration is less than a preset threshold.

In one embodiment, the model training unit 160 is further configured to: train one or more evaluation models on each sub-class data separately, or train a plurality of sub-class data as a whole into one evaluation model.

In one embodiment, the evaluation factors for the coverage of the speech database include one or more of the following: the speaker's gender, the speaker's age, the speaker's accent, the speaker's speaking rate, the speaker's pitch, the speaker's factor or content theme.

The present disclosure also provides a device for measuring the coverage of a speech database, the device comprising: an evaluation model obtaining unit for obtaining each evaluation by using the unsupervised model training method 10 for measuring the coverage of a speech database according to any of the foregoing embodiments The evaluation model of factors, the voice database acquisition unit is used to acquire the voice database to be evaluated, wherein the voice database includes at least one voice; the detection unit is used to detect each voice in the voice database through the evaluation model of the evaluation factor , to obtain the single-factor information entropy of the voice database corresponding to the evaluation factors; the evaluation unit is used to determine the coverage of the voice database according to the single-factor information entropy.

Fig. 5 is a schematic block diagram of an apparatus according to any one of the foregoing embodiments according to an exemplary embodiment. For example, apparatus 200 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, and the like.

5, the apparatus 200 may include one or more of the following components: a processing component 202, a memory 204, a power supply component 206, a multimedia component 208, an audio component 210, an input/output (I/O) interface 212, a sensor component 214, and communication component 216 .

The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or some of the steps of the methods described above. Additionally, processing component 202 may include one or more modules that facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

Memory 204 is configured to store various types of data to support operation at device 200 . Examples of such data include instructions for any application or method operating on the device 200, contact data, phonebook data, messages, pictures, videos, and the like. Memory 204 may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic Disk or Optical Disk.

Power supply assembly 206 provides power to various components of device 200 . Power supply components 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 200 .

The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 208 includes a front-facing camera and/or a rear-facing camera. When the device 200 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.

Audio component 210 is configured to output and/or input audio signals. For example, audio component 210 includes a microphone (MIC) that is configured to receive external audio signals when device 200 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 204 or transmitted via communication component 216 . In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

The I/O interface 212 provides an interface between the processing component 202 and a peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.

Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200 . For example, the sensor assembly 214 can detect the open/closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, and the sensor assembly 214 can also detect a change in the position of the device 200 or a component of the device 200 , the presence or absence of user contact with the device 200 , the orientation or acceleration/deceleration of the device 200 and the temperature change of the device 200 . Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 216 is configured to facilitate wired or wireless communication between apparatus 200 and other devices. Device 200 may access wireless networks based on communication standards, such as WiFi, 2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component 216 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, apparatus 200 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the above method.

In an exemplary embodiment, there is also provided a computer-readable storage medium including instructions, such as a memory 204 including instructions, which are executable by the processor 220 of the apparatus 200 to perform the method described above. For example, the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

FIG. 6 is a block diagram of an electronic device 300 according to an exemplary embodiment. For example, the apparatus 300 may be provided as a server. 6, apparatus 300 includes a processing component 322, which further includes one or more processors, and a memory resource, represented by memory 342, for storing instructions executable by processing component 322, such as an application program. An application program stored in memory 342 may include one or more modules, each corresponding to a set of instructions. Additionally, the processing component 322 is configured to execute instructions to perform the above-described methods.

Device 300 may also include a power supply assembly 326 configured to perform power management of device 300 , a wired or wireless network interface 350 configured to connect device 300 to a network, and an input output (I/O) interface 358 . Device 300 may operate based on an operating system stored in memory 342, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or conventional techniques in the art not disclosed by this disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

It should be understood that the present invention is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from its scope. The scope of the present invention is limited only by the appended claims.

Claims (20)

1. An unsupervised model training method for measuring speech database coverage, the method comprising:

acquiring training data, wherein the training data is voice;

determining one or more evaluation factors for voice database coverage;

dividing the evaluation factor into an adjustable factor or an unadjustable factor based on whether the training data is controllable by parameter adjustment corresponding to the evaluation factor;

determining a clustering algorithm corresponding to each divided evaluation factor, wherein if the evaluation factor is an unadjustable factor, the corresponding clustering algorithm is determined to be a distance-based clustering algorithm, and if the evaluation factor is an adjustable factor, the corresponding clustering algorithm is determined to be an adaptive training algorithm;

classifying the training data respectively through the clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses;

training an evaluation model according to the plurality of subclasses of each of the evaluation factors.

2. The unsupervised model training method for measuring coverage of a speech database according to claim 1, wherein the classifying the training data by the clustering algorithm corresponding to each of the evaluation factors to obtain a plurality of subclasses comprises:

if the evaluation factor is an unadjustable factor, extracting a feature vector of the training data;

and according to the feature vector, dividing the training data into a plurality of subclasses by adopting the distance-based clustering algorithm.

3. The unsupervised model training method for measuring speech database coverage of claim 2, wherein the distance-based clustering algorithm is a K-means clustering algorithm.

4. The unsupervised model training method for measuring coverage of a speech database according to claim 1, wherein the classifying the training data by the clustering algorithm corresponding to each of the evaluation factors to obtain a plurality of subclasses comprises:

if the evaluation factor is an adjustable factor, extracting a feature vector of the training data;

training a Gaussian mixture model through the feature vectors, and labeling the training data;

and according to the marked training data, dividing the training data into a plurality of subclasses.

5. The unsupervised model training method for measuring coverage of a speech database according to claim 4, wherein the training a Gaussian mixture model by the feature vector and labeling the training data comprises:

training a Gaussian mixture model through the feature vectors;

determining control parameters according to the evaluation factors, wherein the control parameters can adjust and control the training data;

traversing all values of the control parameters, and transforming the training data;

obtaining a parameter value when the feature vector of the transformed training data enables the likelihood of the Gaussian mixture model to be maximum;

accumulating likelihood according to the parameter values;

converting the training data according to the parameter values to obtain new training data, and retraining until a stopping condition is reached;

and taking the parameter value corresponding to each training data and enabling the Gaussian mixture model to have the maximum likelihood as the labeled value of the training data.

6. The unsupervised model training method for measuring speech database coverage of claim 5, wherein the stopping condition comprises: and the iteration times reach a preset threshold, or the change rate of the cumulative likelihood and the cumulative likelihood in the last iteration is smaller than the preset threshold.

7. The unsupervised model training method for measuring speech database coverage of claim 1, wherein said training an assessment model according to said plurality of sub-classes of each of said assessment factors comprises: and respectively training one or more evaluation models for each piece of sub-data, or training one evaluation model for a plurality of pieces of sub-data as a whole.

8. The unsupervised model training method for measuring speech database coverage as in claim 1, wherein the evaluation factors for speech database coverage include one or more of: gender of the speaker, age of the speaker, accent of the speaker, speed of speech, pitch, language, capture device, capture environment, pronunciation factors, or content subject.

9. A method for measuring coverage of a voice database, the method comprising obtaining an evaluation model of each evaluation factor by using the unsupervised model training method for measuring coverage of a voice database according to any one of claims 1 to 8;

acquiring a voice database to be evaluated, wherein the voice database comprises at least one voice;

detecting each voice in the voice database through the evaluation model of the evaluation factor to obtain a single-factor information entropy of the voice database corresponding to the evaluation factor;

and determining the coverage degree of the voice database according to the single-factor information entropy.

10. An unsupervised model training apparatus for measuring speech database coverage, the apparatus comprising:

the data acquisition unit is used for acquiring training data, and the training data is voice;

the evaluation factor determining unit is used for determining one or more evaluation factors of the coverage of the voice database;

the dividing unit is used for dividing the evaluation factors into adjustable factors or non-adjustable factors based on whether the training data corresponding to the evaluation factors can be controlled through parameter adjustment;

the algorithm determining unit is used for determining a clustering algorithm corresponding to each divided evaluation factor, wherein when the evaluation factor is an unadjustable factor, the clustering algorithm corresponding to the evaluation factor is determined to be a distance-based clustering algorithm, and when the evaluation factor is an adjustable factor, the clustering algorithm corresponding to the evaluation factor is determined to be an adaptive training algorithm;

the classification unit is used for classifying the training data through the clustering algorithm corresponding to each evaluation factor to obtain a plurality of subclasses;

and the model training unit is used for training an evaluation model according to the plurality of subclasses of each evaluation factor.

11. The unsupervised model training device for measuring coverage of a speech database of claim 10, wherein the classification unit is further configured to:

when the evaluation factor is an unadjustable factor, extracting a feature vector of the training data;

and according to the feature vector, dividing the training data into a plurality of subclasses by adopting the distance-based clustering algorithm.

12. The unsupervised model training device for measuring speech database coverage of claim 11, wherein the distance-based clustering algorithm is a K-means clustering algorithm.

13. The unsupervised model training device for measuring coverage of a speech database of claim 10, wherein the classification unit is further configured to:

when the evaluation factor is an adjustable factor, extracting a feature vector of the training data;

training a Gaussian mixture model through the feature vectors, and labeling the training data;

and according to the marked training data, dividing the training data into a plurality of subclasses.

14. The unsupervised model training device for measuring coverage of a speech database according to claim 13, wherein said training a gaussian mixture model by using the feature vectors and labeling the training data comprises:

training a Gaussian mixture model through the feature vectors;

determining control parameters according to the evaluation factors, wherein the control parameters can adjust and control the training data;

traversing all values of the control parameters, and transforming the training data;

obtaining a parameter value when the feature vector of the transformed training data enables the likelihood of the Gaussian mixture model to be maximum;

accumulating likelihood according to the parameter values;

converting the training data according to the parameter values to obtain new training data, and retraining until a stopping condition is reached;

and taking the parameter value corresponding to each training data and enabling the Gaussian mixture model to have the maximum likelihood as the labeled value of the training data.

15. The unsupervised model training device for measuring speech database coverage of claim 14, wherein the stopping condition comprises: and the iteration times reach a preset threshold, or the change rate of the cumulative likelihood and the cumulative likelihood in the last iteration is smaller than the preset threshold.

16. The unsupervised model training device for measuring speech database coverage of claim 10, wherein the model training device is further configured to: and respectively training one or more evaluation models for each piece of sub-data, or training one evaluation model for a plurality of pieces of sub-data as a whole.

17. The unsupervised model training device for measuring coverage of a voice database of claim 10, wherein the evaluation factors for coverage of the voice database include one or more of: gender of the speaker, age of the speaker, accent of the speaker, speed of speech, pitch, language, capture device, capture environment, pronunciation factors, or content subject.

18. An apparatus for measuring coverage of a voice database, the apparatus comprising an evaluation model obtaining unit, configured to obtain an evaluation model of each evaluation factor by using the unsupervised model training method for measuring coverage of a voice database according to any one of claims 1 to 8;

the system comprises a voice database acquisition unit, a voice database evaluation unit and a voice evaluation unit, wherein the voice database acquisition unit is used for acquiring a voice database to be evaluated, and the voice database comprises at least one voice;

the detection unit is used for detecting each voice in the voice database through the evaluation model of the evaluation factor to obtain the single-factor information entropy of the voice database corresponding to the evaluation factor;

and the evaluation unit is used for determining the coverage of the voice database according to the single-factor information entropy.

19. An electronic device, comprising:

a memory to store instructions; and

a processor for invoking the memory-stored instructions to perform the unsupervised model training method for measuring speech database coverage of any of claims 1-8.

20. A computer-readable storage medium having stored thereon instructions which, when executed by a processor, perform the unsupervised model training method for measuring speech database coverage of any of claims 1 to 8.

Images