CN118152572A - Document clustering and sorting method, system, device and medium based on language large model - Google Patents

Abstract

The present invention discloses a document clustering and sorting method, system, device and medium based on a language macro model, wherein the method comprises: collecting document data for structural processing and preprocessing; inputting document content into a language macro model to obtain vectorized representation; using a clustering algorithm to obtain document clusters and similarity matrices in document clusters for vectorized document content, sorting documents in each document cluster according to the weighted sum of the similarity matrix, and taking the top ten document titles as seed document titles; counting the number of documents of each level in the document cluster, the total number of documents and the weighted sum of the correlation coefficient of the document cluster, and calculating the weighted sum of the three indicators to obtain the final score of each document cluster, and sorting them according to the score; inputting the seed document title and the set prompt into the language macro model to generate a short sentence as the class label of the document cluster. The present invention can make document vectorization more accurate, the class sorting basis more scientific, and the generation of class labels more specific and automatic.

Description

Document clustering and sorting method, system, device and medium based on language large model

Technical Field

The present invention belongs to the field of artificial intelligence, and in particular relates to a document clustering and sorting method, system, device and medium based on a language large model.

Background technique

Traditional document classification requires experts to classify documents based on business scenarios and experience knowledge, usually in alphabetical order, which has a good effect in the era of paper documents. With the continuous development of technology, the degree of document digitization is getting higher and higher, and more and more organizations and individuals are using digital tools and solutions to manage, store and share documents. After the digitization of documents, there are new requirements for document search, which usually needs to focus on certain categories of documents. Moreover, with the increase in the number of documents, manual classification has become very difficult. At the same time, after accumulating a large number of digital documents, information overload and difficulty in processing may occur, making it difficult for users to find the information they need. Therefore, effective management, backup and classified storage are required to ensure the long-term preservation and accessibility of data. If there is no good document management system, data may be lost or unavailable. How to organize, classify and sort these digital documents is particularly important. The rapid development of language large model technology represented by GhatGPT provides a new solution for document classification and sorting.

The current document classification and sorting methods mainly include: 1. Document search classification based on keywords, document sorting based on text similarity: by using keywords to query related documents, the searched documents are input into a pre-trained similarity score prediction model to obtain the similarity score of the candidate document, and the document sorting results of the multiple candidate documents are determined according to the similarity score of each candidate document. 2. Document clustering based on knowledge graph: Based on keyword technology, a keyword-based knowledge graph is generated, and the knowledge graph is divided into several sub-graphs, and the clustering algorithm is used to cluster the documents into events in the knowledge graph. 3. Bidirectional clustering based on word frequency-inverse document matrix: Directly vectorize the document and achieve clustering by calculating the distance between text vectors.

The Chinese patent with the publication number "CN116304071A" discloses a two-layer web document clustering algorithm based on knowledge graph. The invention searches, classifies and sorts relevant documents according to the provided keywords. In the sorting and classification stage, relevant candidate documents are screened from the document library; splicing processing, the sentences related to the keywords in each candidate document are spliced to generate spliced corpus; similarity score prediction, the generated spliced corpus is input into the pre-trained similarity score prediction model to calculate the similarity score of each candidate document; document sorting, according to the similarity score of each candidate document, multiple candidate documents are sorted. The above invention mainly has the following problems: First, the keywords used for classification are determined entirely based on experience. The quality of keyword selection directly determines the accuracy of document classification. When the number of documents is huge, the category coverage is inevitably limited, which will directly affect the accuracy. Second, the classification identification or class label directly uses the selected keywords, resulting in limited ability to summarize the document category; the classification label needs to reflect the main features of all documents in the category as much as possible, generally a phrase or short sentence containing multiple keywords. 3. Computational complexity: concatenating and calculating the similarity scores between documents and queries may require a lot of computing resources and time, especially in the case of large-scale document libraries, which may cause delays and performance issues. 4. The traditional model used in the similarity calculation model has worse model performance than the language model, which may lead to inaccurate ranking.

The Chinese patent with the publication number of "CN111680131B" discloses a semantic-based document clustering method, including: obtaining an input document and preprocessing it; performing word frequency statistics and inverse document frequency calculation on each word contained in the processed input document to construct a word frequency-inverse document matrix; inputting the words used in the word frequency statistics as objects into a pre-stored natural language processing model to obtain a similarity matrix adapted to the word frequency-inverse document matrix; performing semantic propagation on the word frequency-inverse document matrix according to the similarity matrix to obtain a second word frequency-inverse document matrix; performing bidirectional clustering on the second word frequency-inverse document matrix to obtain at least one double cluster cluster, the double cluster cluster includes a document cluster and a word cluster, and labeling each document in the document cluster according to the feature words contained in the word cluster. Unlike the keyword method, the keywords used as class labels in this invention are no longer generated manually, but are generated through clustering. The above invention mainly has the following problems: First, although the use of bidirectional clustering no longer requires manual direct keyword giving, it still belongs to the method of using keywords as class labels, and there is still a lack of using only keywords to express the feature information of document clusters. Second, although the word frequency-inverse document frequency algorithm is simple to implement, easy to understand, and highly explanatory, it is not comprehensive enough to use word frequency to measure the importance of a word in an article, which makes the classification accuracy low. Third, this calculation cannot reflect the position information of the word, and cannot reflect the difference between the same word in different contexts. Fourth, the invention does not involve the sorting of documents after clustering, which is of great significance in the document sorting process.

Summary of the invention

The present invention provides a document clustering and sorting method, system, device and medium based on a language big model, aiming to solve the problems in the prior art that the text vectorization method relies on feature engineering, has weak representation ability, needs to use artificial experience to determine the generation of class labels or screen and generate them in existing texts, and has limited class label expression ability.

In order to solve the above technical problems, the present invention provides a document clustering and sorting method based on a language large model, comprising the following steps:

S1: Collect document data and perform structural processing, the document data includes document content and document information, the document information includes document title and document level, and pre-process the document content that has undergone structural processing.

S2: Input the preprocessed document content into the language model to calculate the vectorized representation of the document content.

S3: A clustering algorithm is used to obtain multiple document clusters for the vectorized document contents. A similarity matrix based on vector distance is obtained for the documents in each document cluster through vector calculation. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix. The top ten documents in the cluster are used as seed documents, and the document title of the seed document is the seed document title.

S4: Use the seed document title to calculate the correlation coefficient within each document cluster to obtain the weighted sum of the correlation coefficients. Based on the document level in the structured document information, count the number of documents of each level in the document cluster. According to the three indicators of the number of documents of each level, the total number of documents and the weighted sum of the correlation coefficient within the document cluster, calculate the weighted sum of the three indicators to obtain the final score of each document cluster, and sort the documents in the document cluster according to the score.

S5: The seed document titles in each document cluster and the set prompt are input into the language model to generate a short sentence summarizing the information features of the document cluster, and the short sentence is used as the class label of the document cluster.

Preferably, in step S3, the documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix as follows:

S31: The vectorized representation set of documents in each document cluster is V = {v ₁ ,v ₂ ,…,v _p }, where p is the total number of documents in the document cluster, and the similarity matrix between the nth document and the mth document in the document cluster is d(v _n ,v _m ). The calculation formula of the similarity matrix d(v _n ,v _m ) is specifically:

Where _vn is the vectorized representation of the nth document in the document cluster, and _vm is the vectorized representation of the mth document in the document cluster.

S32: The calculation formula for the weighted sum of the similarity matrix of the nth document in the document cluster is:

Where _Sn is the weighted sum of the similarity matrix of the nth document, _αi is the weight of the ith similarity matrix, and _vi is the vectorized representation of the ith document in the document cluster.

S33: Sort the documents in each document cluster within the cluster according to the weighted sum of the similarity matrix.

Preferably, in step S4, the seed document is used to calculate the correlation coefficients in each document cluster to obtain the weighted sum of the correlation coefficients. Specifically, the seed document title is input into the language model to calculate the vectorized representation T = {t ₁ , t ₂ , ..., t ₁₀ }, and the calculation formula for the weighted sum of the correlation coefficients of the kth seed document title in each document cluster is specifically:

Where WSCC _k is the weighted sum of the correlation coefficients of the k-th seed document title, β _i is the weight of the i-th similarity matrix, and t _k is the vectorized representation of the k-th seed document title.

The calculation formula for the weighted sum of correlation coefficients within each document cluster is as follows:

Where WSCC is the weighted sum of the correlation coefficients within the document cluster, and _γk is the weight of the kth _WSCCk .

Preferably, the calculation formula for the final score of the document cluster in step S4 is specifically:

CFS＝0.3*DC ₁ +0.2*DC ₂ +0.1*DC+0.4*WSCC

In the formula, CFS is the final score of the document cluster, DC ₁ is the number of documents at level 1, DC ₂ is the number of documents at level 2, DC is the total number of documents, and WSCC is the weighted sum of the correlation coefficients within the document cluster.

On the other hand, the present invention provides a document clustering and sorting system based on a language macro model, comprising a document collection module, a text vectorization module, a document clustering module, a class sorting module and a class label generation module.

The document collection module is used to collect document data and perform structured processing, wherein the document data includes document content and document information, wherein the document information includes document title and document level, and pre-processes the document content that has undergone structured processing.

The text vectorization module is used to input the preprocessed document content into the language model to calculate the vectorized representation of the document content.

The document clustering module is used to obtain multiple document clusters using a clustering algorithm for the vectorized document content. The documents in each document cluster obtain a similarity matrix based on vector distance through vector calculation. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix. The top ten documents in the cluster are used as seed documents, and the document title of the seed document is the seed document title.

The class sorting module is used to use the seed document title to calculate the correlation coefficient within each document cluster to obtain the weighted sum of the correlation coefficient. Based on the document level in the structured document information, the number of documents of each level in the document cluster is counted. According to the three indicators of the number of documents of each level, the total number of documents and the weighted sum of the correlation coefficient within the document cluster, the weighted sum of the three indicators is calculated to obtain the final score of each document cluster, and the document clusters are sorted according to the scores.

The class label generation module is used to generate a short sentence summarizing the information features of the document cluster based on the seed document title in each document cluster and the set prompt input language model, and use the short sentence as the class label of the document cluster.

Preferably, the document clustering module performs intra-cluster sorting on the documents in each document cluster according to the weighted sum of the similarity matrix as follows:

S31: The vectorized representation set of documents in each document cluster is V = {v ₁ ,v ₂ ,…,v _p }, where p is the total number of documents in the document cluster, and the similarity matrix between the nth document and the mth document in the document cluster is d(v _n ,v _m ). The calculation formula of the similarity matrix d(v _n ,v _m ) is specifically:

Where _vn is the vectorized representation of the nth document in the document cluster, and _vm is the vectorized representation of the mth document in the document cluster.

S32: The calculation formula for the weighted sum of the similarity matrix of the nth document in the document cluster is:

Where S _n is the weighted sum of the similarity matrix of the nth document, α _i is the weight of the ith similarity matrix, and _ci is the vectorized representation of the ith document in the document cluster.

S33: Sort the documents in each document cluster within the cluster according to the weighted sum of the similarity matrix.

Preferably, the class ranking module uses seed documents to calculate the correlation coefficients in each document cluster to obtain the weighted sum of correlation coefficients as follows: the seed document title is input into the language model to obtain a vectorized representation T = {t ₁ , t ₂ , ..., t ₁₀ }, and the calculation formula for the weighted sum of correlation coefficients of the kth seed document title in each document cluster is as follows:

Where WSCC _k is the weighted sum of the correlation coefficients of the k-th seed document title, β _i is the weight of the i-th similarity matrix, and t _k is the vectorized representation of the k-th seed document title.

The calculation formula for the weighted sum of correlation coefficients within each document cluster is as follows:

Where WSCC is the weighted sum of the correlation coefficients within the document cluster, and _γk is the weight of the kth _WSCCk .

Preferably, the calculation formula for the final score of the document cluster in the class ranking module is specifically:

CFS＝0.3*DC ₁ +0.2*DC ₂ +0.1*DC+0.4*WSCC

In the formula, CFS is the final score of the document cluster, DC ₁ is the number of documents at level 1, DC ₂ is the number of documents at level 2, DC is the total number of documents, and WSCC is the weighted sum of the correlation coefficients within the document cluster.

On the other hand, the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the document clustering and sorting method based on the language large model as described in any embodiment of the present invention is implemented.

On the other hand, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the document clustering and sorting method based on a large language model as described in any embodiment of the present invention is implemented.

Compared with the prior art, the present invention has the following technical effects:

1. Document vectorization is more accurate: The present invention no longer requires preprocessing of the original document, such as word segmentation. Document vectorization based on the generative language large model can dynamically vectorize the document, thereby solving the problem of polysemy, making subsequent process steps based on vectorization more reasonable and the results more accurate.

2. The basis for class sorting is more scientific: The present invention not only relies on the distance between document vectors, but also takes into account multi-dimensional indicators such as document source level and the number of documents in the class, and comprehensively determines the class sorting through a more scientific and appropriate sorting basis.

3. Automatic and specific generation of class labels: The present invention uses a large language model combined with the top ten document titles in a document cluster to generate a title representing the document cluster, which can express and highlight the characteristics of the document cluster in a more specific manner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is an overall flow chart of the document clustering and sorting method based on the language macro model of the present invention;

FIG. 2 is an overall structural diagram of the document clustering and sorting system based on the language macro model described in the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in combination with specific embodiments of the present application and with reference to the accompanying drawings.

Embodiment 1

This embodiment provides a document clustering and sorting method based on a language big model, which uses the ELMO method in the language big model to vectorize text; generates class labels based on short sentences generated by the language big model, rather than pre-given or clustered keywords; after using the clustering algorithm, the document sorting not only uses the document vector distance, but also combines the number of documents, document level and other features for comprehensive sorting. This embodiment can improve the accuracy of document clustering and the level of organization, classification and sorting. The class label representation capability is more specific, and the document sorting rules in the class are more reasonable. Referring to Figure 1, the following steps are included:

S1: Collect document data and perform structured processing for structured storage. The collected document data can be various types of rules and regulations documents issued by the company, etc. The document data includes document content and document information, and the document information includes document title and document level. The structured storage usually stores the document level information in the document data into a relational database for structured storage. The document content that has been structured is preprocessed, and the preprocessing includes deleting and cleaning special symbols and punctuation that have no practical meaning, HTML tags of web documents, document data decryption, document data desensitization, etc. Different from the traditional technical route based on keywords or word frequency-inverse document algorithm, since this embodiment uses a large language model for text vectorization, preprocessing operations such as word segmentation, removal of low-frequency words, and stop words are no longer required.

S2: Input the preprocessed document content into the language model to calculate the vectorized representation of the document content. The language model can use online interfaces such as ChatGPT, or localized language model services. The preprocessed document content is calculated using the embedding module of the language model to obtain the vectorized representation of the document content. Compared with the word frequency-inverse document algorithm, the vectorization of the language model is based on the ELMO method, which is more accurate, has a higher dimension, contains more information features, and can solve the problem of polysemy by combining text context information.

S3: A clustering algorithm is used to obtain multiple document clusters for the vectorized document content. The clustering algorithm includes K-means clustering, DBSCA, Hierarchical Clustering, etc. The appropriate algorithm or model can be selected according to the actual scenario and requirements.

K-means clustering is an iterative clustering analysis algorithm. As the preferred implementation of this embodiment, this embodiment uses K-means clustering to obtain document clusters. Specifically, its implementation steps are: first, pre-divide the documents into K groups, randomly select K documents (vectorized representation) as the initial cluster centers, then calculate the distance between each document and each cluster center, and assign each document to the cluster center with the closest distance (Euclidean distance) to it. A document cluster is represented by the cluster center documents and the documents assigned to them. Whenever a sample document is assigned, the cluster center of the category will be recalculated based on the existing document vectors in the class. This process will be repeated until a certain termination condition is met. The termination condition can be that no (or a set minimum number) objects are reassigned to different clusters, no (or a set minimum number) cluster centers change again, and the sum of squared errors is locally minimized.

DBSCA, the algorithm uses the concept of density-based clustering, which requires that the number of objects (points or other spatial objects) contained in a certain area in the clustering space is not less than a given threshold. The significant advantages of the DBSCAN algorithm are fast clustering speed and the ability to effectively handle noise points and find spatial clusters of arbitrary shapes. However, when the density of spatial clusters is uneven and the distance between clusters varies greatly, the clustering quality is poor.

Hierarchical Clustering creates a hierarchical nested cluster tree by calculating the similarity between data points of different categories. The smaller the distance, the higher the similarity. In the cluster tree, the original data points of different categories are the lowest level of the tree, and the top level of the tree is the root node of a cluster. There are two ways to create a cluster tree: bottom-up merging and top-down splitting.

The documents in each document cluster are vector-calculated to obtain a similarity matrix based on vector distance. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix. The top ten documents in the cluster are used as seed documents, where a document cluster, i.e., a document category, contains multiple documents, and the document title of the seed document is the seed document title.

As a preferred implementation of this embodiment, in step S3, the documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix as follows:

S31: The vectorized representation set of documents in each document cluster is V = {v ₁ ,v ₂ ,…,v _p }, where p is the total number of documents in the document cluster, and the similarity matrix between the nth document and the mth document in the document cluster is d(v _n ,v _m ). The calculation formula of the similarity matrix d(v _n ,v _m ) is specifically:

Where _vn is the vectorized representation of the nth document in the document cluster, and _vm is the vectorized representation of the mth document in the document cluster.

S32: The calculation formula for the weighted sum of the similarity matrix of the nth document in the document cluster is:

Where _Sn is the weighted sum of the similarity matrices of the nth document, _αi is the weight of the ith similarity matrix, and _vi is the vectorized representation of the ith document in the document cluster.

S33: Sort the documents in each document cluster within the cluster according to the weighted sum of the similarity matrix.

S4: Use the seed document title to calculate the correlation coefficient within each document cluster to obtain the weighted sum of the correlation coefficients. Based on the document level in the structured document information, count the number of documents of each level in the document cluster. According to the three indicators of the number of documents of each level, the total number of documents and the weighted sum of the correlation coefficient within the document cluster, calculate the weighted sum of the three indicators to obtain the final score of each document cluster, and sort the documents in the document cluster according to the score.

As a preferred implementation of this embodiment, in step S4, the seed document is used to calculate the correlation coefficient within each document cluster to obtain the weighted sum of the correlation coefficients. Specifically, the seed document title is input into the language model to calculate the vectorized representation T = {t ₁ , t ₂ , ..., t ₁₀ }. The calculation formula for the weighted sum of the correlation coefficients of the k-th seed document title in each document cluster is specifically:

Where WSCC _k is the weighted sum of the correlation coefficients of the k-th seed document title, β _i is the weight of the i-th similarity matrix, and t _k is the vectorized representation of the k-th seed document title.

The calculation formula for the weighted sum of correlation coefficients within each document cluster is as follows:

Where WSCC is the weighted sum of the correlation coefficients within the document cluster, and _γk is the weight of the kth _WSCCk .

As a preferred implementation of this embodiment, the calculation formula for the final score of the document cluster in step S4 is specifically:

CFS＝0.3*DC ₁ +0.2*DC ₂ +0.1*DC+0.4*WSCC

In the formula, CFS is the final score of the document cluster, DC ₁ is the number of documents at level 1, DC ₂ is the number of documents at level 2, DC is the total number of documents, and WSCC is the weighted sum of the correlation coefficients within the document cluster.

S5: Input the seed document titles in each document cluster and the set prompt into the language model to generate a short sentence summarizing the information characteristics of the document cluster, and use the short sentence as the class label of the document cluster. The seed document title of the document cluster obtained based on the language model and the above steps is the top ten weighted sum of the correlation coefficients with other documents in the document cluster, which is more representative of the information characteristics of the document cluster than the pre-set keywords. The prompt can be set to something similar to the following: The following ten document titles are given one or more words or phrases according to similarity, and the characteristics of these titles are summarized. The titles are as follows... (list the ten document titles obtained).

Embodiment 2

Correspondingly, referring to FIG. 2 , this embodiment provides a document clustering and sorting system based on a large language model, including a document collection module, a text vectorization module, a document clustering module, a class sorting module and a class label generation module.

The document collection module is used to collect document data and perform structured processing. The document data includes document content and document information. The document information includes document title and document level. The structured document content is pre-processed. This module is used to implement the function of step S1 in Example 1, which will not be repeated here.

The text vectorization module is used to input the preprocessed document content into the language model to calculate the vectorized representation of the document content. This module is used to implement the function of step S2 in Example 1, which will not be repeated here.

The document clustering module is used to use a clustering algorithm to obtain multiple document clusters for the vectorized document content. The documents in each document cluster obtain a similarity matrix based on vector distance through vector calculation. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix. The top ten documents in the cluster are used as seed documents. The document title of the seed document is the seed document title. This module is used to implement the function of step S3 in Example 1, which will not be repeated here.

The class sorting module is used to use the seed document title to calculate the correlation coefficient within each document cluster to obtain the weighted sum of the correlation coefficients. Based on the document level in the structured document information, the number of documents of each level in the document cluster is counted. According to the three indicators of the number of documents of each level, the total number of documents and the weighted sum of the correlation coefficient within the document cluster, the weighted sum of the three indicators is calculated to obtain the final score of each document cluster, and the document clusters are sorted according to the scores. This module is used to implement the function of step S4 in Example 1, which will not be repeated here.

The class label generation module is used to generate a short sentence summarizing the information characteristics of the document cluster based on the seed document title in each document cluster and the set prompt input language model, and use the short sentence as the class label of the document cluster. This module is used to implement the function of step S5 in Example 1, which will not be repeated here.

Embodiment 3

This embodiment provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the document clustering and sorting method based on a large language model as described in any embodiment of the present invention is implemented.

Embodiment 4

This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the document clustering and sorting method based on a large language model as described in any embodiment of the present invention is implemented.

In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And/or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and/or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character "/" generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims (10)

1. A document clustering and sorting method based on a large language model, characterized in that it comprises the following steps: S1: collecting document data and performing structural processing, wherein the document data includes document content and document information, wherein the document information includes a document title and a document level, and pre-processing the document content after structural processing; S2: Input the preprocessed document content into the language model to calculate the vectorized representation of the document content; S3: A clustering algorithm is used to obtain multiple document clusters for the vectorized document contents. A similarity matrix based on vector distance is obtained for the documents in each document cluster through vector calculation. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix. The top ten documents in the cluster are taken as seed documents, and the document title of the seed document is the seed document title. S4: using the seed document title to calculate the correlation coefficient within each document cluster to obtain the weighted sum of the correlation coefficients, based on the document level in the structured document information, counting the number of documents of each level in the document cluster, and calculating the weighted sum of the three indicators of the number of documents of each level, the total number of documents and the weighted sum of the correlation coefficient within the document cluster to obtain the final score of each document cluster, and sorting the documents in the document cluster according to the score; S5: The seed document titles in each document cluster and the set prompt are input into the language model to generate a short sentence summarizing the information features of the document cluster, and the short sentence is used as the class label of the document cluster. 2. The document clustering method based on a language large model according to claim 1 is characterized in that the step S3 of sorting the documents in each document cluster according to the weighted sum of the similarity matrix is specifically as follows: S31: The vectorized representation set of documents in each document cluster is V = {v ₁ ,v ₂ ,…,v _p }, where p is the total number of documents in the document cluster, and the similarity matrix between the nth document and the mth document in the document cluster is d(v _n ,v _m ). The calculation formula of the similarity matrix d(v _n ,v _m ) is specifically: Where _vn is the vectorized representation of the nth document in the document cluster, and _vm is the vectorized representation of the mth document in the document cluster; S32: The calculation formula for the weighted sum of the similarity matrix of the nth document in the document cluster is: Where S _n is the weighted sum of the similarity matrix of the nth document, α _i is the weight of the i-th similarity matrix, and c _i is the vectorized representation of the i-th document in the document cluster; S33: Sort the documents in each document cluster within the cluster according to the weighted sum of the similarity matrix. 3. The document clustering method based on the language big model according to claim 2 is characterized in that the step S4 uses the seed document to calculate the correlation coefficient in each document cluster to obtain the weighted sum of the correlation coefficients: the seed document title is input into the language big model to calculate the vectorized representation T = {t ₁ , t ₂ , ..., t ₁₀ }, and the calculation formula of the weighted sum of the correlation coefficients of the k-th seed document title in each document cluster is specifically: Where WSCC _k is the weighted sum of the correlation coefficients of the k-th seed document title, β _i is the weight of the i-th similarity matrix, and t _k is the vectorized representation of the k-th seed document title; The calculation formula for the weighted sum of correlation coefficients within each document cluster is as follows: Where WSCC is the weighted sum of the correlation coefficients within the document cluster, and _γk is the weight of the kth WSCC _k0 . 4. The document clustering method based on a language large model according to claim 3 is characterized in that the calculation formula for the final score of the document cluster in step S4 is specifically: CFS＝0.3*DC ₁ +0.2*DC ₂ +0.1*DC+0.4*WSCC In the formula, CFS is the final score of the document cluster, DC ₁ is the number of documents at level 1, DC ₂ is the number of documents at level 2, DC is the total number of documents, and WSCC is the weighted sum of the correlation coefficients within the document cluster. 5. A document clustering and sorting system based on a large language model, characterized by comprising a document collection module, a text vectorization module, a document clustering module, a class sorting module and a class label generation module; The document collection module is used to collect document data and perform structural processing, wherein the document data includes document content and document information, wherein the document information includes document title and document level, and pre-process the document content after structural processing; The text vectorization module is used to input the pre-processed document content into the language model to calculate the vectorized representation of the document content; The document clustering module is used to obtain multiple document clusters by using a clustering algorithm on the document content that has been vectorized. The documents in each document cluster are vectorized to obtain a similarity matrix based on vector distance. The documents in each document cluster are sorted within the cluster according to the weighted sum of the similarity matrix. The top ten documents in the cluster are used as seed documents, and the document title of the seed document is the seed document title. The class sorting module is used to calculate the correlation coefficient within each document cluster using the seed document title to obtain the weighted sum of the correlation coefficient, count the number of documents of each level in the document cluster based on the document level in the structured document information, calculate the weighted sum of the three indicators of the number of documents of each level, the total number of documents and the weighted sum of the correlation coefficient within the document cluster, obtain the final score of each document cluster, and sort the document clusters according to the scores; The class label generation module is used to generate a short sentence summarizing the information features of the document cluster based on the seed document title in each document cluster and the set prompt input language model, and use the short sentence as the class label of the document cluster. 6. The document clustering system based on the language big model according to claim 5 is characterized in that the document clustering module sorts the documents in each document cluster according to the weighted sum of the similarity matrix in the cluster as follows: S31: The vectorized representation set of documents in each document cluster is V = {v ₁ ,v ₂ ,…,v _p }, where p is the total number of documents in the document cluster, and the similarity matrix between the nth document and the mth document in the document cluster is d(v _n ,v _m ). The calculation formula of the similarity matrix d(v _n ,v _m ) is specifically: Where _vn is the vectorized representation of the nth document in the document cluster, and _vm is the vectorized representation of the mth document in the document cluster; S32: The calculation formula for the weighted sum of the similarity matrix of the nth document in the document cluster is: Where S _n is the weighted sum of the similarity matrix of the nth document, α _i is the weight of the i-th similarity matrix, and _vi is the vectorized representation of the i-th document in the document cluster; S33: Sort the documents in each document cluster within the cluster according to the weighted sum of the similarity matrix. 7. The document clustering system based on the language big model according to claim 6 is characterized in that the class ranking module uses the seed document to calculate the correlation coefficient in each document cluster to obtain the weighted sum of the correlation coefficients as follows: the seed document title is input into the language big model to calculate the vectorized representation T = {t ₁ , t ₂ , ..., t ₁₀ }, and the calculation formula of the weighted sum of the correlation coefficients of the k-th seed document title in each document cluster is as follows: Where WSCC _k is the weighted sum of the correlation coefficients of the k-th seed document title, β _i is the weight of the i-th similarity matrix, and t _k is the vectorized representation of the k-th seed document title; The calculation formula for the weighted sum of correlation coefficients within each document cluster is as follows: Where WSCC is the weighted sum of the correlation coefficients within the document cluster, and _γk is the weight of the kth _WSCCk . 8. The document clustering system based on the language large model according to claim 7 is characterized in that the calculation formula of the final score of the document cluster in the class ranking module is specifically: CFS＝0.3*DC ₁ +0.2*DC ₂ +0.1*DC+0.4*WSCC In the formula, CFS is the final score of the document cluster, DC ₁ is the number of documents at level 1, DC ₂ is the number of documents at level 2, DC is the total number of documents, and WSCC is the weighted sum of the correlation coefficients within the document cluster. 9. An electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the document clustering and sorting method based on a large language model as described in any one of claims 1 to 4 when executing the computer program. 10. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the document clustering and sorting method based on a large language model as described in any one of claims 1 to 4 is implemented.