CN114596863A - Interactive voiceprint clustering method and system, electronic equipment and storage medium - Google Patents

Abstract

The invention relates to the technical field of voiceprint clustering, in particular to an interactive voiceprint clustering method, system, electronic equipment and storage medium. The method comprises the following steps: extracting a feature vector of the voice file, and clustering based on a preset clustering threshold value; auditing each type of clustered voice files to confirm the final number of people contained in the voice files; re-clustering according to the number of confirmed people, calculating a mean value based on the voiceprint features of each class to serve as the central features of the voiceprint features of the class, solving the similarity between each feature and the central features in the class, and marking the documents which are lower than the similarity threshold as documents which are difficult to classify based on a preset similarity threshold; and reviewing the hard classified files again and re-allocating the hard classified files to correct classes. The invention combines the advantages of the algorithm and the human by combining the human and the algorithm and combining the advantages of the algorithm and the human by an interactive method, solves the problem of the insufficiency of the algorithm and further improves the final clustering effect.

Description

An interactive voiceprint clustering method, system, electronic device and storage medium

technical field

The invention relates to the technical field of voiceprint clustering, in particular to an interactive voiceprint clustering method, system, electronic device and storage medium.

Background technique

Voiceprint clustering is to group the voice files of the same person into one category. In the prior art, an algorithm is used to extract voiceprint features in speech, and by feature comparison, those with high similarity are classified into one category. The clustering method is generally to set the similarity threshold or set the number of people as the cluster termination condition.

The accuracy of clustering is limited by the algorithm itself. Taking the number of people as the termination condition will have a better clustering effect, but the algorithm cannot accurately obtain the number of people. However, using the similarity threshold as the termination condition is relatively ineffective.

SUMMARY OF THE INVENTION

Aiming at the problem that the voiceprint clustering method in the prior art in which the similarity threshold is set or the number of people is set as the clustering termination condition is limited in accuracy and cannot accurately obtain the number of people information, the present invention provides an interactive voiceprint clustering method. Class methods, systems, electronic devices, and storage media, using human interaction to confirm the classification of people, and to deal with difficult-to-classify speech problems.

To achieve the above purpose, the embodiments of the present invention provide the following technical solutions:

In a first aspect, in an embodiment provided by the present invention, an interactive voiceprint clustering method is provided, including the following steps:

Extract the feature vector of the voice file, and perform clustering based on the preset clustering threshold;

Auditing each type of voice file after clustering to confirm the final number of people contained in the voice file;

Re-cluster according to the confirmed number of people, and calculate the mean value based on the voiceprint features of each class as the central feature of the voiceprint feature of this class, find the similarity between each feature in the class and the central feature, and based on the preset similarity Threshold, marking files below the similarity threshold as difficult to classify;

The hard-to-categorize files are reviewed again and reassigned to the correct category.

In some embodiments provided by the present invention, the feature vector of the voice file is extracted by a voiceprint extraction algorithm, and the method for extracting the feature vector of the voice file includes the following steps:

Get the voice file to be extracted;

Perform voice recognition on the acquired voice file to obtain the audio feature of the voice file;

Input the obtained audio features into the model to obtain the feature vector of the voice file.

In some embodiments provided by the present invention, voiceprint feature extraction is performed on the voice file, and an ivector or xvector algorithm is used to extract the feature vector of the voice file to obtain the feature vector of the voice file.

In some embodiments provided by the present invention, when clustering is performed based on a preset clustering threshold, the similarity calculation is performed on different feature vectors extracted from the voice file, wherein the similarity between the different feature vectors is calculated according to the similarity algorithm of cosine or plda. Spend.

In some embodiments provided by the present invention, when performing clustering based on a preset clustering threshold, AHC clustering or SC clustering method is used to perform clustering.

In some embodiments provided by the present invention, the preset clustering threshold is a threshold result confirmed by testing. Before confirming the threshold results, a clustering test is also performed through the given test file, and the threshold with the best effect is selected as the default threshold used later.

In some embodiments provided by the present invention, the auditing of each type of voice file after clustering is performed manually, and judgment is made by listening to the voice of the speaker of the recording file, and the same person but different types are merged into the same category.

In some embodiments provided by the present invention, when the hard-to-categorize files are reviewed again, the deficiency of the algorithm is compensated for the manual review again, and the files are re-assigned to the correct category.

In a second aspect, in another embodiment provided by the present invention, an interactive voiceprint clustering system is provided, including:

A feature extraction unit, used for performing speech recognition on the acquired voice file to extract feature vectors;

a clustering unit for clustering the extracted feature vectors based on a preset clustering threshold;

The similarity comparison unit is used to take the mean value of the voiceprint features of each class after re-clustering the number of people as the central feature of the voiceprint feature of this class, to obtain the similarity between each feature in the class and the central feature, and based on the preset similarity threshold comparison, marking documents below the similarity threshold as difficult to classify; and

The review unit is used for reviewing each type of voice files after clustering to confirm the final number of people included in the voice files, and is also used for reviewing the difficult-to-classify files again and reassigning them to the correct category.

In a third aspect, in another embodiment provided by the present invention, an electronic device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements interaction when loading and executing the computer program The steps of the voiceprint clustering method of the formula.

In a fourth aspect, in yet another embodiment provided by the present invention, a storage medium is provided, which stores a computer program, and when the computer program is loaded and executed by a processor, the interactive voiceprint clustering method is implemented. step.

The technical scheme provided by the invention has the following beneficial effects:

The interactive voiceprint clustering method, system, electronic device and storage medium of the present invention extract feature vectors of voice files, perform manual review after clustering, and classify different types of voice files after clustering but belong to the same type. People are merged into the same class, and the final number of people is confirmed according to the final number of merged classes. The algorithm is used to mark the difficult-to-classify files according to the similarity, and the difficult-to-classify files are reassigned to the correct class during manual review. Through the combination of people and algorithms, the advantages of algorithms and people are combined in an interactive way, and the shortcomings of algorithms are solved, thereby improving the effect of final clustering.

These and other aspects of the invention will be more clearly understood from the description of the following embodiments. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the attached image:

FIG. 1 is a flowchart of an interactive voiceprint clustering method according to the present invention.

FIG. 2 is a flowchart of feature vector extraction in an interactive voiceprint clustering method according to an embodiment of the present invention.

FIG. 3 is a system block diagram of an interactive voiceprint clustering system in an embodiment of the present invention.

FIG. 4 is a structural block diagram of an electronic device in an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

In some of the processes described in the description and claims of the present invention and the above-mentioned drawings, various operations are included in a specific order, but it should be clearly understood that these operations may not be in accordance with the order in which they appear herein. For execution or parallel execution, the sequence numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these flows may include more or fewer operations, and these operations may be performed sequentially or in parallel.

The technical solutions in the exemplary embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the exemplary embodiments of the present invention. Obviously, the described exemplary embodiments are only part of the embodiments of the present invention, rather than All examples. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

FIG. 1 is a flowchart of an interactive voiceprint clustering method provided by the present invention.

Referring to FIG. 1 , the present invention provides an interactive voiceprint clustering method, which utilizes interaction with people to confirm the number of people to classify, and to deal with the problem of difficult-to-classify speech.

In this embodiment, the interactive voiceprint clustering method includes the following steps:

S1, extract the feature vector of the voice file, and perform clustering based on a preset clustering threshold;

S2. Review each type of voice file after clustering to confirm the final number of people included in the voice file;

S3. Re-cluster according to the confirmed number of people, and calculate the mean value based on the voiceprint features of each class as the central feature of the voiceprint feature of this class, obtain the similarity between each feature in the class and the central feature, and based on the preset Similarity threshold, marking files below the similarity threshold as difficult to classify;

S4. Review the hard-to-categorize files again, and reassign them to correct categories.

The interactive voiceprint clustering method of this embodiment is divided into the above-mentioned four main steps to perform, wherein, the clustering of voice files and the marking of difficult-to-classify files are both voiceprint clustering operations completed by algorithms, and the The auditing of each type of voice files after the clustering and the re-examining of the hard-to-classify files are all done manually. Combining the respective advantages of algorithms and people, it solves the shortcomings of the algorithm, thereby improving the effect of the final clustering.

In step S1 of this embodiment, the feature vector of the voice file is extracted by a voiceprint extraction algorithm. Referring to FIG. 2 , the method for extracting the feature vector of the voice file includes the following steps:

S101, obtaining a voice file to be extracted;

S102, performing speech recognition on the acquired voice file to obtain an audio feature of the voice file;

S103. Input the obtained audio feature into the model, and obtain the feature vector of the voice file.

In this embodiment, the algorithm for extracting voiceprint adopts the current mainstream ivector, xvector, and the like. The specific steps are other front-end features of the audio file, such as mfcc or fbank. In this embodiment, voiceprint feature extraction is performed on the voice file, and the ivector or xvector algorithm is used to extract the feature vector of the voice file, and then input into the model, and finally the feature vector is obtained. .

In this embodiment, during the clustering operation, the similarity calculation of different feature vectors is performed according to the plda algorithm or the cosine algorithm, wherein the plda algorithm or the cosine algorithm is an existing resident similarity calculation algorithm. In this embodiment, the clustering similarity algorithm includes but is not limited to the plda algorithm or the cosine algorithm.

Therefore, when clustering is performed based on the preset clustering threshold, the similarity calculation is performed on different feature vectors extracted from the voice file, wherein the similarity is calculated according to the similarity algorithm of cosine or plda.

In this embodiment, when performing clustering based on a preset clustering threshold, AHC clustering or SC clustering method is used to perform clustering. It should be noted that, when performing clustering based on a preset clustering threshold, the clustering method includes but is not limited to AHC clustering or SC clustering method.

In this embodiment, the preset clustering threshold is a threshold result confirmed through testing. Before confirming the threshold results, a clustering test is also performed through the given test file, and the threshold with the best effect is selected as the default threshold used later.

When clustering is performed based on the preset clustering threshold, the preset clustering threshold will adopt a higher value, so that the same person is in each category. After the above clustering, the number of clusters is usually larger than the actual number of clusters, but the number of clusters after clustering is much smaller than the number of voice files. The clustering operation can greatly save the time of artificial clustering and improve the This improves the efficiency of voiceprint clustering operations and facilitates subsequent manual review.

For example, in a voiceprint extraction algorithm according to an embodiment of the present invention, the preset clustering threshold is 0.7, and the higher threshold is 0.8. There are 20 voice files for clustering, which are divided into five categories. Threshold clustering, each class is basically the same person's voice. Then the manual only needs to listen to any one of each category, there are 5 categories, so only need to listen to 5 voices, and then merge the same category. Here, it is assumed that the two categories are manually merged, and the last 20 files are divided into 4 categories. In this embodiment, different algorithms calculate the similarity in different ways, so the higher threshold is also relatively speaking.

In step S2 of this embodiment, the auditing of each type of voice file after clustering is performed manually, and judgment is made by listening to the voice of the speaker of the recording file, and the same person but different types are merged into the same type.

When manually reviewing, according to listening to the voice files of each category, the same person will be merged into the same category, and the final number of people will be confirmed according to the final number of merged categories; due to the high threshold in the previous step, the same person will be divided into two groups. People, this step uses manual methods to further merge to confirm the number of people. The higher threshold in the previous step is used to confirm the number of people. Compared with the normal threshold, the result will be more classified, and each class will be more pure, which is better. Combined with manual operation.

In this embodiment, the manual review is judged by listening to the voice of the speaker of the recording file. There will be many clustered files, but the actual number of speakers will not be many, so it is feasible to judge manually. For example, for two voice files, the similarity score is low due to noise, differences in acquisition equipment, or defects in the algorithm itself, but for humans, this problem can be compensated, and it can be well judged whether the two voices are the same person.

Re-cluster according to the number of people confirmed in step S2, and calculate the mean value based on the voiceprint features of each class as the central feature of the class. In this embodiment, the voiceprint feature is a one-dimensional vector calculated by an algorithm that can characterize the voiceprint. When calculating the mean as the central feature of the class, for example, there are three files, and the extracted voiceprint vectors are A, B, and C. Then the central feature is 1/3*(A+B+C).

When calculating the similarity between each feature and the central feature in the class, the similarity calculation method is the same as that of step S2. For example, cosine or plda algorithm can be used. If the similarity is lower than the preset low threshold, it is a difficult-to-classify file, or referred to as low-similarity files.

In step S3, since the clustering effect based on the number of people is generally better than the clustering effect based on the similarity threshold, the clustering is performed again after confirming the number of people to improve the accuracy of the clustering. At the same time, this step adopts the forced classification of the number of people, and some with low similarity will be classified into one category. This category may be misclassified due to algorithm problems and needs to be further confirmed later.

Therefore, when the difficult-to-classify voice files are manually reviewed again, step S3 marks the voices with low similarity as difficult to classify, and in step S4, artificial methods are used to make up for the deficiencies of the algorithm, thereby further improving the classification accuracy.

In this embodiment, when the hard-to-categorize files are reviewed again, the deficiencies of the algorithm are compensated for by manual review, and are reassigned to the correct category.

It should be understood that although the above is described in a certain order, these steps are not necessarily performed sequentially in the above order. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, a part of the steps in this embodiment may include multiple steps or multiple stages. These steps or stages are not necessarily executed and completed at the same time, but may be executed at different times, and the execution sequence of these steps or stages is also different. It is necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a portion of the steps or stages within the other steps.

In one embodiment, as shown in FIG. 3 , an embodiment of the present invention provides an interactive voiceprint clustering system, the system includes a feature extraction unit 100 , a clustering unit 200 , a similarity comparison unit 300 and an auditing unit 400.

Wherein, the feature extraction unit 100 is configured to perform voice recognition on the acquired voice file to extract feature vectors. In this embodiment, the algorithm for extracting voiceprint adopts the current mainstream ivector, xvector, and the like. Wherein, in this embodiment, voiceprint feature extraction is performed on the voice file, and the ivector or xvector algorithm is used to extract the feature vector of the voice file, and then input into the model to finally obtain the feature vector.

The clustering unit 200 is configured to cluster the extracted feature vectors based on a preset clustering threshold. When performing clustering based on a preset clustering threshold, the clustering unit 200 adopts AHC clustering or SC clustering method for clustering.

It should be noted that, when performing clustering based on a preset clustering threshold, the clustering method includes but is not limited to AHC clustering or SC clustering method.

The similarity comparison unit 300 is used for re-clustering the number of people and the voiceprint feature mean of each class as the central feature of the voiceprint feature of this class, to obtain the similarity of each feature and the central feature in the class, and based on the preset The similarity threshold is compared, and the files below the similarity threshold are marked as difficult to classify.

When the similarity comparison unit 300 performs the clustering operation, different feature vectors perform similarity calculation according to the plda algorithm or the cosine algorithm, wherein the plda algorithm or the cosine algorithm is an existing resident similarity calculation algorithm. In this embodiment, the clustering similarity algorithm includes but is not limited to the plda algorithm or the cosine algorithm.

When performing clustering based on a preset clustering threshold, the similarity calculation is performed on different feature vectors extracted from the speech file, wherein the similarity is calculated according to the similarity algorithm of cosine or plda.

The review unit 400 is used to review each type of voice file after the clustering to confirm the final number of people contained in the voice file, and is also used to review the difficult-to-classify files again and reassign them to the correct category. .

The auditing unit 400 usually conducts manual auditing, and when manually auditing, according to listening to the voice files of each category, merge different categories but belong to the same person into the same category, and confirm the final number of people according to the final number of categories merged; The step threshold is higher, and the same person will be divided into two people. In this step, the number of people is confirmed by further merging by means of manual methods. The higher threshold value in the previous step is combined to confirm the number of people. The result will be more classified than the normal threshold value. The class is also more pure, which is better combined with manual operations.

In this embodiment, the interactive voiceprint clustering system adopts the steps of the above-mentioned interactive voiceprint clustering method. Therefore, in this embodiment, the interactive voiceprint clustering system has The operation process will not be described in detail.

In one embodiment, FIG. 4 shows a structural block diagram of an electronic device according to an embodiment of the present invention. In a possible design, as shown in FIG. 4 , an electronic device 600 is provided in an embodiment of the present invention. The electronic device 600 includes a memory 601 and a processor 602 , and a computer program is stored in the memory 601 to process the The device 602 is configured to execute the computer program stored in the memory 601 . The memory 601 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 602 to implement the steps in the above method embodiments:

Extract the feature vector of the voice file, and perform clustering based on the preset clustering threshold;

Auditing each type of voice file after clustering to confirm the final number of people contained in the voice file;

Re-cluster according to the confirmed number of people, and calculate the mean value based on the voiceprint features of each class as the central feature of the voiceprint feature of this class, find the similarity between each feature in the class and the central feature, and based on the preset similarity Threshold, marking files below the similarity threshold as difficult to classify;

The hard-to-categorize files are reviewed again and reassigned to the correct category.

It should be noted that, according to the embodiments of the present invention, the methods described above with reference to the accompanying drawings may be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a readable medium thereof, the computer program containing program code for performing the methods of the Figures. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion, and/or installed from a removable medium.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the diagram or block diagram may represent a unit, segment, or portion of code that contains one or more functions for implementing the specified logical function. executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

An embodiment of the present invention provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps in the foregoing method embodiments:

Extract the feature vector of the voice file, and perform clustering based on the preset clustering threshold;

Auditing each type of voice file after clustering to confirm the final number of people contained in the voice file;

Re-cluster according to the confirmed number of people, and calculate the mean value based on the voiceprint features of each class as the central feature of the voiceprint feature of this class, find the similarity between each feature in the class and the central feature, and based on the preset similarity Threshold, marking files below the similarity threshold as difficult to classify;

The hard-to-categorize files are reviewed again and reassigned to the correct category.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing the relevant hardware through a computer program represented by computer instructions, and the computer program can be stored in a non-volatile computer readable In the storage medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any reference to memory, storage, database or other media used in the various embodiments provided in this application may include at least one of non-volatile and volatile memory.

Non-volatile memory may include read-only memory, magnetic tape, floppy disk, flash memory, or optical memory, among others. Volatile memory may include random access memory or external cache memory. By way of illustration and not limitation, RAM may take various forms, such as static random access memory or dynamic random access memory, among others.

To sum up, the interactive voiceprint clustering method, system, electronic device and storage medium of the present invention extract feature vectors of voice files, perform manual review after clustering, and classify each type of voice file after clustering into For the merging of different categories but belonging to the same person, the final number of people is confirmed according to the final number of merged categories. The algorithm is used to mark the difficult-to-classify files according to the similarity, and manual review is performed to reassign the difficult-to-classify files to the correct class. Through the combination of people and algorithms, the advantages of algorithms and people are combined in an interactive way, and the shortcomings of algorithms are solved, thereby improving the effect of final clustering.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (10)

1. An interactive voiceprint clustering method, comprising:

extracting a feature vector of the voice file, and clustering based on a preset clustering threshold value;

auditing each type of clustered voice files to confirm the final number of people contained in the voice files;

re-clustering according to the number of confirmed people, calculating a mean value based on the voiceprint features of each class to serve as the central features of the voiceprint features of the class, solving the similarity between each feature and the central features in the class, and marking the documents which are lower than the similarity threshold as documents which are difficult to classify based on a preset similarity threshold;

and reviewing the hard classified files again and re-allocating the hard classified files to correct classes.

2. The interactive voiceprint clustering method of claim 1 wherein: the feature vector of the voice file is extracted through a voiceprint extraction algorithm, and the method for extracting the feature vector of the voice file comprises the following steps:

acquiring a voice file to be extracted;

carrying out voice recognition on the obtained voice file to obtain the audio characteristics of the voice file;

and inputting the obtained audio features into a model to obtain feature vectors of the voice file.

3. The interactive voiceprint clustering method of claim 2 wherein: and extracting the voiceprint features of the voice file, and extracting the feature vector of the voice file by adopting an vector or xvector algorithm to obtain the feature vector of the voice file.

4. The interactive voiceprint clustering method of claim 1 wherein: and when clustering is carried out based on a preset clustering threshold, carrying out similarity calculation on different feature vectors extracted from the voice file, wherein the similarity is calculated by the different feature vectors according to a similarity calculation method of cosine or plda.

5. The interactive voiceprint clustering method of claim 1 wherein: and when clustering is carried out based on a preset clustering threshold value, clustering is carried out by adopting an AHC clustering or SC clustering method.

6. The interactive voiceprint clustering method of claim 5 wherein: the preset clustering threshold is a threshold result confirmed by testing.

7. The interactive voiceprint clustering method of claim 6 wherein: and auditing is manually performed when each type of clustered voice files are audited, and different types but the same type are merged into the same type by judging through hearing the voice of the speaker of the record file.

8. An interactive voiceprint clustering system, characterized by: the interactive voiceprint clustering system adopts the interactive voiceprint clustering method of any one of claims 1 to 7 to improve the final clustering effect; the interactive voiceprint clustering system comprises:

the characteristic extraction unit is used for carrying out voice recognition on the acquired voice file to extract a characteristic vector;

the clustering unit is used for clustering the extracted feature vectors based on a preset clustering threshold value;

the similarity comparison unit is used for taking the mean value of the voiceprint features of each class after people are clustered again as the central features of the voiceprint features of the class, solving the similarity between each feature in the class and the central features, comparing the similarity based on a preset similarity threshold, and marking the tags which are lower than the similarity threshold as documents which are difficult to classify; and

and the auditing unit is used for auditing each type of clustered voice files to confirm the final number of people contained in the voice files, and is also used for auditing the files difficult to classify again and redistributing the files into correct classes.

9. An electronic device comprising a memory and a processor, the memory storing a computer program, wherein the steps of the method of any one of claims 1 to 7 are implemented when the computer program is loaded and executed by the processor.

10. A storage medium storing a computer program, characterized in that the computer program is loaded by a processor and executed to implement the steps of the method according to any of claims 1 to 7.

Images