CN113033641B - Semi-supervised classification method for high-dimensional data - Google Patents

Abstract

The invention discloses a semi-supervised classification method for high-dimensional data, and relates to the field of artificial intelligence semi-supervised learning. The method mainly overcomes the influence of data noise and redundant features on a model in high-dimensional data in the manufacturing industry, integrates subspace learning, graph construction and classifier training into a unified framework, and achieves a better classification effect. The method comprises the following steps: 1) Inputting a training data set; 2) Normalizing the data; 3) Initializing parameters and variables; 4) Subspace learning; 5) Constructing a graph; 6) Training a classifier; 7) Repeating the steps 4) -6) circularly until the algorithm is converged; 8) Classifying the test samples; 9) And obtaining the classification accuracy. The invention completes the construction of the graph from two low-dimensional spaces, namely the label space and the subspace, effectively relieves the interference of noise data and redundant characteristics on an algorithm model, ensures the quality of the graph and improves the classification effect.

Description

A semi-supervised classification method for high-dimensional data

technical field

The invention relates to the technical field of artificial intelligence semi-supervised learning, in particular to a high-dimensional data semi-supervised classification method.

Background technique

With the advent of the intelligent era, some traditional manufacturing industries are gradually moving closer to intelligent manufacturing. In view of the large amount of data generated by the manufacturing industry, using intelligent decision-making methods to optimize production, sales, service and other processes is one of the main problems that intelligent manufacturing needs to face. The manufacturing industry often accumulates a large amount of data in the process of development. However, in general, these big data are not all labeled. In the face of a large amount of data and a small number of labels, if we want to use a fully supervised classification algorithm to model and analyze the data to learn certain patterns of these data, we often cannot achieve satisfactory results. So, how to learn the inherent patterns of data from a large amount of data and a small number of labels? One of the solutions is to try to label a large amount of training data, but this is expensive and requires a lot of manpower and material resources. Obviously, a better solution is to start directly from the algorithm and model, and design an algorithm model so that it can learn a classification model with better performance and strong generalization ability from data with only a small number of labels. The semi-supervised classification algorithm is exactly such an algorithm model. It uses a small number of labeled samples and a large number of unlabeled samples to learn and classify data, thus saving the overhead of manually labeling training samples. Therefore, the semi-supervised classification algorithm has important research significance, has attracted the research and exploration of a large number of scientific researchers in recent years, and has a good application prospect in industry.

Graph-based semi-supervised classification algorithms are one of the more popular research directions in the semi-supervised field in recent years, because they often have better performance. Such algorithms are based on the assumption that the data should be in a manifold space, and the distribution of samples should be smooth enough. The so-called smoothness means that the closer samples, that is, samples with higher similarity, their labels should be as identical as possible. In this type of algorithm, it is usually necessary to construct a graph to represent the similarity between samples, and then obtain the smoothness term between samples, and then combine the loss function, regularization term and smoothness term as the overall objective function of the model , by optimizing the objective function to solve the classifier parameters, so that the final trained classifier not only has a small classification loss on the labeled sample, but also has a classification result on all samples (including labeled samples and unlabeled samples) Also smooth enough.

However, some current graph-based semi-supervised classification algorithms are not well suited for high-dimensional data scenarios in manufacturing. For example, the data in the manufacturing industry often contains missing values and data noise, which will interfere with the construction of the graph and have a certain impact on the performance of the model. Another problem is that when dealing with high-dimensional data in the manufacturing industry, due to the influence of data noise and redundant features, graph-based semi-supervised classification algorithms often cannot perform well.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and propose a semi-supervised classification method for high-dimensional data, which can effectively alleviate the influence of data noise and redundant features in high-dimensional data on the model, and construct the graph The process and classifier training process are integrated into a unified framework, which significantly improves the classification performance in semi-supervised classification scenarios.

In order to achieve the above object, the technical solution provided by the present invention is: a semi-supervised classification method for high-dimensional data, comprising the following steps:

1) Input the training data set, which is a high-dimensional data set;

2) Normalize the data, eliminate the influence of different feature dimensions, and increase the speed of subsequent optimization learning at the same time;

子空间投影矩阵

其中d为样本的特征数，c为样本类别数，

表示d行c列的实数矩阵；初始化W的低秩分解矩阵

其中

表示c行c列的实数矩阵；初始化相似度矩阵

参数矩阵

其中n为样本数量，

表示n行n列的实数矩阵；初始化偏置向量

其中

表示c行1列的实数矩阵；3) Initialize the regression matrix

subspace projection matrix

Where d is the number of features of the sample, c is the number of sample categories,

Represents a real matrix of d rows and c columns; initializes the low-rank decomposition matrix of W

in

Represents a real matrix of c rows and c columns; initialize the similarity matrix

parameter matrix

where n is the sample size,

Represents a real matrix of n rows and n columns; initializes the bias vector

in

Represents a real number matrix with c rows and 1 column;

4) Subspace learning: According to the proposed subspace learning objective function, the optimal solution of low-rank decomposition matrix B, parameter matrix C and subspace projection matrix A is derived; since the proposed objective function involves multiple optimization variables, alternate optimization is used The method, update B, C, A iteratively, gradually optimize, improve the quality of the subspace, and then learn the optimal subspace that expresses the essential characteristics of the sample;

5) Comprehensively learn the sample similarity matrix from two aspects: the sample subspace and the sample label space; define the samples as the nodes of the graph, define the similarity between samples as the edges of the graph, and the learning process of the sample similarity matrix is Graph construction process;

6) On the basis of step 4) subspace learning and step 5) similarity matrix learning, learn a semi-supervised linear regression classifier, that is, learn a regression matrix W and a bias vector b;

7) Perform step 4) to step 6) in a loop, and iteratively learn each variable until convergence; when convergence, the three processes of subspace learning, graph construction and classifier learning have also obtained a joint optimal solution;

8) Classify the test samples, assuming that the input test sample is x and the number of sample categories is c, then the prediction label predict(x) is:

Among them, (W ^T x+b) _i represents the i-th element of the vector (W ^T x+b);

9) Calculate the classification accuracy rate: input the label of the test sample, compare it with the prediction result, and calculate the final classification accuracy rate. Since the test sample used is high-dimensional data and there is no unbalanced data, only the classification accuracy rate is used. Judge the effect.

In step 2), the data normalization step is: obtain the maximum value X(r) _max and the minimum value X(r) _min corresponding to the r-th row of data, and convert the d-th row of data according to the following formula:

为第r行第i个数据，

为更新之后的数据，n为数据集中样本数量，d为样本的特征数，i∈{1,2,...,n},r∈{1,2,...,d}。in,

is the i-th data in row r,

is the updated data, n is the number of samples in the data set, d is the feature number of the sample, i∈{1,2,...,n}, r∈{1,2,...,d}.

为全零矩阵；初始化W的低秩分解矩阵

为全零矩阵；初始化相似度矩阵

和参数矩阵

为全零矩阵；初始化偏置向量

为全零向量；初始化子空间投影矩阵A＝qf(R)为正交矩阵；其中，

是一个随机矩阵，每个元素都在区间[0,1]，qf(·)表示的是QR分解。In step 3), the initialization method is: initialize the regression matrix

is an all-zero matrix; initialize the low-rank decomposition matrix of W

is an all-zero matrix; initialize the similarity matrix

and parameter matrix

is an all-zero matrix; initialize the bias vector

Be all zero vector; Initialization subspace projection matrix A=qf (R) is an orthogonal matrix; Wherein,

is a random matrix, each element is in the interval [0,1], and qf(·) represents the QR decomposition.

In step 4), the subspace learning process is as follows:

Define the objective function of subspace learning as:

表示矩阵的F-范数，

为样本矩阵，

表示d行n列的实数矩阵，

为回归矩阵，

为子空间投影矩阵，

是W的低秩分解矩阵，

是参数矩阵；α,θ,β为可调节的参数；Among them, tr( ) is the trace of the matrix,

represents the F-norm of the matrix,

is the sample matrix,

Represents a real number matrix with d rows and n columns,

is the regression matrix,

is the subspace projection matrix,

is the low-rank decomposition matrix of W,

is a parameter matrix; α, θ, β are adjustable parameters;

Calculate the partial derivative of the objective function with respect to B, C, and A respectively, and the update formula of each variable can be obtained; then, update each variable according to the following requirements:

a. Update B according to the formula B=A ^T W;

b. Update C according to the formula C=(X ^T AA ^T X+I) ^-1 X ^T AA ^T X;

c. cyclically update the subspace projection matrix A according to the following formula: A _t+1 =qf(A _t +G), until convergence;

Among them, I is the identity matrix, t represents the t-th iteration, A _t represents the value of A in the t-th iteration, A _t+1 represents the value of A in the t+1-th iteration, and G represents the gradient of the objective function , G=-2(X(αL+θ(IC)(IC) ^T )X ^T A-βWB ^T ), qf(·) means QR decomposition.

In step 5), the construction process of the graph is: jointly learn the similarity matrix from the two aspects of the sample label space and the sample subspace, and define the objective function of the similarity matrix learning as:

表示矩阵的F-范数，

是回归矩阵，

是子空间投影矩阵，

是样本矩阵，

表示d行n列的实数矩阵，

是相似度矩阵，

是拉普拉斯矩阵，并且L＝D-S，D是一个对角矩阵，

D_ii表示矩阵D第i行第i列上的元素，S_ij表示相似度矩阵S第i行第j列上的元素；参数λ是正则项的权重；Among them, tr( ) is the trace of the matrix,

represents the F-norm of the matrix,

is the regression matrix,

is the subspace projection matrix,

is the sample matrix,

Represents a real number matrix with d rows and n columns,

is the similarity matrix,

is a Laplacian matrix, and L=DS, D is a diagonal matrix,

D _ii represents the element on row i, column i of matrix D, and S _ij represents the element on row i, column j of similarity matrix S; parameter λ is the weight of the regular term;

Let the number of neighbors of each sample be k, that is, the similarity between each sample and k neighbor samples is not 0, and the others are all 0; let x _i , x _j denote the i-th and j-th samples respectively; define e _ij is the sum of the Euclidean distance between x _i and x _j in the subspace and the Euclidean distance in the label space, then the calculation formula of e _ij is as follows:

Then, according to the solution of the objective function, the update formula of the similarity matrix S can be obtained:

Among them, the intermediate variable

In step 6), the semi-supervised linear regression classifier learning process is as follows:

The basic objective function that defines a semi-supervised linear regression classifier is:

表示矩阵的F-范数，

是回归矩阵，

是样本矩阵，

表示d行n列的实数矩阵，

是偏置向量，

是样本的标签矩阵，参数γ是正则项权重，

是对角矩阵，如果样本x_i是带标签样本，则U_ii＝1,否则U_ii＝0，其中U_ii表示矩阵U第i行第i列上的元素；Among them, tr( ) is the trace of the matrix,

represents the F-norm of the matrix,

is the regression matrix,

is the sample matrix,

Represents a real number matrix with d rows and n columns,

is the bias vector,

Is the label matrix of the sample, and the parameter γ is the weight of the regular term,

is a diagonal matrix, if the sample x _i is a labeled sample, then U _ii =1, otherwise U _ii =0, where U _ii represents the element on the i-th row and i-th column of the matrix U;

Combining the above objective function with the objective function of subspace learning in step 4) and the objective function of similarity matrix learning in step 5), the final objective function is obtained:

是参数矩阵，

是子空间投影矩阵，

是相似度矩阵；

是W的低秩分解矩阵，

是拉普拉斯矩阵，并且L＝D-S，D是一个对角矩阵，

D_ii表示矩阵D第i行第i列上的元素，S_ij表示相似度矩阵S第i行第j列上的元素；参数α,θ,β是调整各个项重要程度的权重；Among them, Loss=tr((W ^T X+b1 ^T -Y)U(W ^T X+b1 ^T -Y) ^T ),

is the parameter matrix,

is the subspace projection matrix,

is the similarity matrix;

is the low-rank decomposition matrix of W,

is a Laplacian matrix, and L=DS, D is a diagonal matrix,

D _ii represents the element on row i, column i of matrix D, S _ij represents the element on row i, column j of similarity matrix S; parameters α, θ, β are the weights to adjust the importance of each item;

Calculate the partial derivative of the above final objective function with respect to W and b respectively, and the update formulas of W and b are as follows:

W＝[XU _c X ^T +αXLX ^T +β(I-AA ^T )+γI] ^-1 XU _c Y ^T

Among them, the intermediate variable

Then, update W and b according to the above update formula, and the learning process of the semi-supervised linear regression classifier can be completed.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

The invention has a solid mathematical theory foundation, and has great advantages in accuracy, stability and robustness. First, constructing graphs from two low-dimensional spaces, label space and subspace, can overcome the influence of redundant features in high-dimensional manufacturing data. Moreover, the graph constructed from the two spaces is also more robust and better adapts to the characteristics of unstable data distribution. Second, in the process of learning the subspace, the low-rank property of the regression matrix is used, which makes it easier for the subspace to distinguish samples of different categories. Third, the three processes of subspace learning, graph construction, and classifier training are integrated into a unified framework. Through cyclical alternate optimization, the three processes promote each other to achieve a joint optimal solution, which significantly improves the overall performance of the algorithm framework. learning ability.

Description of drawings

Fig. 1 is a schematic diagram of the logic flow of the present invention.

Fig. 2 is the accuracy rate contrast table of the present invention and traditional semi-supervised classification algorithm and the semi-supervised classification algorithm based on graph, SSCNGC is the abbreviation of the method of the present invention, and the number bold is effect best, and data format is " accuracy rate ± standard Difference".

detailed description

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

As shown in Figure 1, the high-dimensional data semi-supervised classification method provided by this embodiment includes the following steps:

1) The input training data set is a high-dimensional data set.

2) Normalize the data, eliminate the influence of different feature dimensions, and at the same time increase the speed of subsequent optimization learning; where, the data normalization step is: obtain the maximum value X(r) _max and The minimum value is X(r) _min , and the data in row d is converted according to the following formula:

为第r行第i个数据，

为更新之后的数据，n为数据集中样本数量，d为样本的特征数，i∈{1,2,...,n},r∈{1,2,...,d}。in,

is the i-th data in row r,

is the updated data, n is the number of samples in the data set, d is the feature number of the sample, i∈{1,2,...,n}, r∈{1,2,...,d}.

子空间投影矩阵

其中d为样本的特征数，c为样本类别数，

表示d行c列的实数矩阵；初始化W的低秩分解矩阵

其中

表示c行c列的实数矩阵；初始化相似度矩阵

参数矩阵

其中n为样本数量，

表示n行n列的实数矩阵；初始化偏置向量

其中

表示c行1列的实数矩阵；3) Initialize the regression matrix

subspace projection matrix

Where d is the number of features of the sample, c is the number of sample categories,

Represents a real matrix of d rows and c columns; initializes the low-rank decomposition matrix of W

in

Represents a real matrix of c rows and c columns; initialize the similarity matrix

parameter matrix

where n is the sample size,

Represents a real matrix of n rows and n columns; initializes the bias vector

in

Represents a real number matrix with c rows and 1 column;

为全零矩阵；初始化W的低秩分解矩阵

为全零矩阵；初始化相似度矩阵

和参数矩阵

为全零矩阵；初始化偏置向量

为全零向量；初始化子空间投影矩阵A＝qf(R)为正交矩阵；其中，

是一个随机矩阵，每个元素都在区间[0,1]，qf(·)表示的是QR分解。The initialization method is: initialize the regression matrix

is an all-zero matrix; initialize the low-rank decomposition matrix of W

is an all-zero matrix; initialize the similarity matrix

and parameter matrix

is an all-zero matrix; initialize the bias vector

Be all zero vector; Initialization subspace projection matrix A=qf (R) is an orthogonal matrix; Wherein,

is a random matrix, each element is in the interval [0,1], and qf(·) represents the QR decomposition.

4) Subspace learning: According to the proposed subspace learning objective function, the optimal solution of low-rank decomposition matrix B, parameter matrix C and subspace projection matrix A is derived; since the proposed objective function involves multiple optimization variables, alternate optimization is used The method, update B, C, A iteratively, gradually optimize, improve the quality of the subspace, and then learn the optimal subspace that expresses the essential characteristics of the sample;

The subspace learning process is as follows:

Define the objective function of subspace learning as:

表示矩阵的F-范数，

为样本矩阵；α,θ,β为可调节的参数；Among them, tr( ) is the trace of the matrix,

represents the F-norm of the matrix,

is the sample matrix; α, θ, β are adjustable parameters;

Calculate the partial derivative of the objective function with respect to B, C, and A respectively, and the update formula of each variable can be obtained; then, update each variable according to the following requirements:

a. Update B according to the formula B=A ^T W;

b. Update C according to the formula C=(X ^T AA ^T X+I) ^-1 X ^T AA ^T X;

c. cyclically update the subspace projection matrix A according to the following formula: A _t+1 =qf(A _t +G), until convergence;

Among them, I is the identity matrix, t represents the t-th iteration, A _t represents the value of A in the t-th iteration, A _t+1 represents the value of A in the t+1-th iteration, and G represents the gradient of the objective function , G=-2(X(αL+θ(IC)(IC) ^T )X ^T A-βWB ^T ), qf(·) means QR decomposition.

5) Comprehensively learn the sample similarity matrix from two aspects: the sample subspace and the sample label space; define the samples as the nodes of the graph, define the similarity between samples as the edges of the graph, and the learning process of the sample similarity matrix is Graph construction process;

The construction process of the graph is as follows: jointly learn the similarity matrix from two aspects of the sample label space and the sample subspace, and define the objective function of the similarity matrix learning as:

Among them, the parameter λ is the weight of the regular term;

Let the number of neighbors of each sample be k, that is, the similarity between each sample and k neighbor samples is not 0, and the others are all 0; let x _i , x _j denote the i-th and j-th samples respectively; define e _ij is the sum of the Euclidean distance between x _i and x _j in the subspace and the Euclidean distance in the label space, then the calculation formula of e _ij is as follows:

Then, according to the solution of the objective function, the update formula of the similarity matrix S can be obtained:

Among them, the intermediate variable

6) On the basis of step 4) subspace learning and step 5) similarity matrix learning, learn a semi-supervised linear regression classifier, that is, learn a regression matrix W and a bias vector b;

The semi-supervised linear regression classifier learning process is as follows:

The basic objective function that defines a semi-supervised linear regression classifier is:

是对角矩阵，如果样本x_i是带标签样本，则U_ii＝1,否则U_ii＝0，其中U_ii表示矩阵U第i行第i列上的元素；in,

is a diagonal matrix, if the sample x _i is a labeled sample, then U _ii =1, otherwise U _ii =0, where U _ii represents the element on the i-th row and i-th column of the matrix U;

Combining the above objective function with the objective function of subspace learning in step 4) and the objective function of similarity matrix learning in step 5), the final objective function is obtained:

Among them, Loss=tr((W ^T X+b1 ^T -Y)U(W ^T X+b1 ^T -Y) ^T ); parameters α, θ, β are weights to adjust the importance of each item;

Calculate the partial derivative of the above final objective function with respect to W and b respectively, and the update formulas of W and b are as follows:

W＝[XU _c X ^T +αXLX ^T +β(I-AA ^T )+γI] ^-1 XU _c Y ^T

Among them, the intermediate variable

Then, update W and b according to the above update formula, and the learning process of the semi-supervised linear regression classifier can be completed.

7) Perform step 4) to step 6) in a loop, and iteratively learn each variable until convergence; when convergence, the three processes of subspace learning, graph construction and classifier learning will obtain a joint optimal solution.

8) Classify the test sample, assuming that the input test sample is x, then the prediction label predict(x) is:

Among them, (W ^T x+b) _i represents the ith element of the vector (W ^T x+b).

9) Calculate the classification accuracy rate: input the label of the test sample, compare it with the prediction result, and calculate the final classification accuracy rate. Since the test sample used is high-dimensional data and there is no unbalanced data, only the classification accuracy rate is used. Judge the effect.

Fig. 2 is the accuracy rate contrast table of the present invention and traditional semi-supervised classification algorithm and the semi-supervised classification algorithm based on graph, SSCNGC is the abbreviation of the method of the present invention, and the number bold is effect best, and data format is " accuracy rate ± standard Difference". As can be seen from the figure, in the experiments of 16 high-dimensional data sets, the present invention has achieved the highest accuracy rate on 15 of them, and has achieved an improvement of more than 5% on 9 data sets. It shows that the present invention has stronger advantages compared with the traditional semi-supervised algorithm.

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (5)

1. A semi-supervised classification method for high-dimensional data is characterized by comprising the following steps:

1) Inputting a training data set which is a high-dimensional data set;

2) Normalizing the data, eliminating the influence of different characteristic dimensions, and simultaneously improving the speed of subsequent optimization learning;

3) Initializing a regression matrix

Subspace projection matrix

Where d is the number of features of the sample, c is the number of sample classes,

a real matrix representing d rows and c columns; initializing a low rank decomposition matrix of W

Wherein

A real number matrix representing c rows and c columns; initializing a similarity matrix

Parameter matrix

Where n is the number of samples in the sample,

a real number matrix representing n rows and n columns; initialization bias directionMeasurement of

Wherein

A real matrix representing c rows and 1 columns;

4) Subspace learning: deriving an optimal solution of a low-rank decomposition matrix B, a parameter matrix C and a subspace projection matrix A according to the proposed subspace learning objective function; because the proposed objective function relates to a plurality of optimization variables, B, C and A are iteratively updated by an alternate optimization method, the optimization is carried out step by step, the subspace quality is improved, and the optimal subspace for expressing the essential characteristics of the sample is learned; the subspace learning process is as follows:

the objective function defining the subspace learning is:

wherein tr (-) is the trace of the matrix,

the F-norm of the matrix is represented,

in the form of a matrix of samples,

a matrix of real numbers representing d rows and n columns,

in the form of a regression matrix,

for the projection matrix of the subspace,

is a low-rank decomposition matrix of W,

is a parameter matrix; alpha, theta, beta are adjustable parameters;

the target function is subjected to partial derivatives of B, C and A respectively, and an updating formula of each variable can be obtained; next, each variable is updated as required:

a. according to the formula B = A ^T W updates B;

b. according to the formula C = (X) ^T AA ^T X+I) ^-1 X ^T AA ^T Updating C by X;

c. circularly updating the subspace projection matrix A according to the following formula: a. The _t+1 ＝qf(A _t + G) until convergence;

wherein, I is an identity matrix, t represents the t-th iteration, A _t Represents the value of A in the t-th iteration, A _t+1 Denotes the value of A in the t +1 th iteration, G denotes the gradient of the objective function, G = -2 (X (. Alpha.L + theta (I-C)) ^T )X ^T A-βWB ^T ) Qf (·) denotes QR decomposition;

5) Comprehensively learning a sample similarity matrix from two aspects of a sample subspace and a sample label space; defining samples as nodes of the graph, defining the similarity among the samples as edges of the graph, wherein the learning process of the sample similarity matrix is the construction process of the graph;

6) Learning a semi-supervised linear regression classifier, namely learning a regression matrix W and a bias vector b, on the basis of the subspace learning in the step 4) and the similarity matrix learning in the step 5);

7) Circularly performing the step 4) to the step 6), and iteratively learning each variable until convergence; when convergence occurs, joint optimal solutions are obtained through three processes of subspace learning, graph construction and classifier learning;

8) Classifying the test samples, assuming that the input test sample is x and the number of sample classes is c, predicting (x) of the prediction label is as follows:

wherein (W) ^T x+b) _i Represents a vector (W) ^T x + b) th element;

9) Calculating the classification accuracy: and inputting labels of the test samples, comparing the labels with the prediction result, and calculating the final classification accuracy.

2. The semi-supervised classification method for high-dimensional data as recited in claim 1, wherein: in step 2), the data normalization step is: obtaining the maximum value X (r) corresponding to the r row data _max And minimum X (r) _min And converting the d-th row of data according to the following formula:

wherein,

for the ith data of the r-th row,

for the data after update, n is the number of samples in the dataset, d is the number of features of the samples, i ∈ {1, 2.., n }, r ∈ {1, 2.., d }.

3. The semi-supervised classification method for high-dimensional data as recited in claim 1, wherein: in step 3), the initialization method is as follows: initializing a regression matrix

Is an all-zero matrix; initializing a low rank decomposition matrix of W

Is an all-zero matrix; initializing a similarity matrix

And parameter matrix

Is an all-zero matrix; initializing a bias vector

Is an all-zero vector; initializing subspace projection matrices a = qf (R) as orthogonal matrices; wherein,

is a random matrix with each element in the interval 0,1]And qf (·) denotes QR decomposition.

4. The semi-supervised classification method for high-dimensional data as recited in claim 1, wherein: in step 5), the construction process of the graph is as follows: the similarity matrix is jointly learned from two aspects of the sample label space and the sample subspace, and an objective function for defining the similarity matrix learning is as follows:

wherein tr (-) is the trace of the matrix,

the F-norm of the matrix is represented,

is a regression matrix, and the regression matrix is,

is a subspace projectionThe shadow matrix is a matrix of the image,

is a matrix of samples of the sample to be sampled,

a matrix of real numbers representing d rows and n columns,

is a matrix of the degree of similarity (or similarity matrix),

is a laplacian matrix and L = D-S, D is a diagonal matrix,

D _ii representing the elements of matrix D at row i and column i, S _ij Representing the elements in the ith row and the jth column of the similarity matrix S; the parameter λ is the weight of the regularization term;

setting the number of neighbors of each sample as k, namely, the similarity between each sample and the k neighbor samples is not 0, and the similarity is 0; let x _i ,x _j Respectively representing the ith and the j th samples; definition e _ij Is x _i And x _j Sum of Euclidean distance in subspace and Euclidean distance in tag space, then e _ij The calculation formula of (c) is as follows:

then, according to the solution of the objective function, an update formula of the similarity matrix S can be obtained:

wherein the intermediate variable

5. The semi-supervised classification method for high-dimensional data as recited in claim 1, wherein: in step 6), the learning process of the semi-supervised linear regression classifier is as follows:

the basic objective function that defines a semi-supervised linear regression classifier is:

where tr (-) is the trace of the matrix,

the F-norm of the matrix is represented,

is a regression matrix, and the regression matrix is,

is a matrix of samples of the sample to be sampled,

a matrix of real numbers representing d rows and n columns,

is a vector of the offset to the offset,

is the label matrix of the sample, the parameter gamma is the regularization term weight,

is a diagonal matrix if sample x _i If it is a sample with a label, U _ii =1, otherwise U _ii =0, wherein U _ii Represents the matrix U at the ith row and ith columnThe element (b);

combining the objective function with the objective function learned by the subspace in the step 4) and the objective function learned by the similarity matrix in the step 5) to obtain a final objective function:

wherein Loss = tr ((W) ^T X+b1 ^T -Y)U(W ^T X+b1 ^T -Y) ^T )，

Is a matrix of parameters that is a function of,

is a projection matrix of the subspace,

is a similarity matrix;

is a low-rank decomposition matrix of W,

is a laplacian matrix and L = D-S, D is a diagonal matrix,

D _ii representing the elements of matrix D at row i and column i, S _ij Representing the elements in the ith row and the jth column of the similarity matrix S; the parameters alpha, theta and beta are weights for adjusting the importance degree of each item;

and respectively solving the partial derivatives of the final objective function to W and b to obtain the update formulas of W and b as follows:

W＝[XU _c X ^T +αXLX ^T +β(I-AA ^T )+γI] ^-1 XU _c Y ^T

wherein the intermediate variable

And then, updating W and b according to the updating formula, and finishing the learning process of the semi-supervised linear regression classifier.

Images