CN114817258B - Network source data compliance verification method, device and system - Google Patents

Abstract

The invention discloses a network source data compliance verification method, device and system, belonging to the technical field of data verification, wherein the method comprises the following steps: acquiring related files which are input or uploaded from a power plant in advance, performing OCR text image recognition on the related files, extracting network source data information and storing the network source data information into a database; extracting network source data information to be checked from the database; performing verification rule configuration on the network source data information; carrying out compliance verification according to the verification rule information, and screening out non-compliance data; performing data verification on the data subjected to the verification, and generating a verification result table for operation and maintenance personnel to review the verification result and prompt the power plant to correct the problem; inputting the non-compliance data into a pre-constructed modification scheme model to generate a new modification scheme; according to the invention, the new rectifying and modifying scheme is provided for the selection of the client by rapidly positioning the missing data and the non-compliance data, so that various data compliance problems in the network source coordination operation and maintenance can be well found and solved.

Description

A network source data compliance verification method, device and system

Technical Field

The present invention relates to a network source data compliance verification method, device and system, belonging to the technical field of data verification.

Background Art

As the scale of power grid business data and the complexity of data quality monitoring rules continue to increase, the processing capacity of the existing data quality monitoring system based on traditional data management and computing platforms has encountered serious bottlenecks, low processing efficiency, and difficulty in quickly completing data quality monitoring and verification, making it increasingly difficult to meet the needs of daily production management and business decision-making.

In the existing technology, data verification through machine learning can only learn some existing knowledge, and there may be a large amount of missing data in the network source data, and machine learning cannot give results. Therefore, the accuracy and efficiency of network source data processing by existing technologies cannot achieve the expected goals, and the network source coordination design has more than 30 database table structures and more than a dozen document data, including project approval documents, project filing documents, grid-connected dispatching agreements, industrial and commercial business licenses (three certificates in one), power purchase and sales contracts, certificates for work, power generation business licenses, excitation test reports, Pss test reports, speed regulation test reports, phase advance test reports, primary frequency regulation test reports, AGC test reports, AVC test reports, standard systems, regulations and specifications, etc. The content is relatively complex, and manual verification cannot well complete the operation and maintenance of the entire system.

Summary of the invention

The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a network source data compliance verification method, device and system that can quickly locate missing data and non-compliant data and provide new rectification plans for customers to choose and use.

To achieve the above object, the present invention is implemented by adopting the following technical solutions:

In one aspect, the present invention provides a method for verifying compliance of network source data, comprising:

Obtain relevant documents pre-entered or uploaded from the power plant, perform OCR text image recognition on the relevant documents, extract network source data information and store it in the database;

Extracting network source data information that needs to be verified from the database;

Performing verification rule configuration on the network source data information;

Perform compliance verification according to the verification rule information to filter out non-compliant data;

The non-compliant data is input into a pre-built rectification plan model to generate a new rectification plan.

Furthermore, the configuration of verification rules for the network source data information includes:

Different verification rules are configured for different data tables in the network source data information to form a verification rule library, including one or more of missing seals or signatures, qualified ranges of different data, and validity periods of documents.

Furthermore, the verification rule configuration of the network source data information provides a business rule dynamic configuration management method, including:

Based on the rule engine framework - Drools, establish a network source business rule library and configure corresponding business rules according to different business categories;

Obtain verification request information, and use the RETE algorithm to perform rule matching in the rule engine according to the request information to obtain one or more verification logic rules. When multiple logic rules are obtained, prioritize the multiple logic rules and select the optimal verification logic rule;

According to the optimal business logic rules, the corresponding services are called and executed.

Furthermore, when the compliance check is performed according to the check rule information, multi-threaded check processing is performed according to different data tables.

Furthermore, it also includes: data verification of data after compliance verification, generating a verification result table for operation and maintenance personnel to review the verification results and urge the power plant to rectify the problems.

Furthermore, the construction of the rectification plan model includes:

Using a deep learning method based on convolutional neural networks and a top-down supervised learning approach, a rectification plan model is obtained by training the rectification plans for previous data non-compliance.

Furthermore, it also includes: when generating rectification plans, returning multiple rectification plans in order of matching degree for user selection.

In a second aspect, the present invention provides a network source data compliance verification device, comprising:

A network source data information storage unit is used to obtain relevant files pre-entered or uploaded from the power plant, perform OCR text image recognition on the relevant files, extract network source data information and store it in a database;

A network source data information acquisition unit, used to extract network source data information that needs to be verified from the database;

A verification rule configuration unit, used to configure verification rules for the network source data information;

A compliance verification unit, used to perform compliance verification according to the verification rule information and filter out non-compliant data;

The rectification plan generating unit is used to input the non-compliant data into a pre-built rectification plan model to generate a new rectification plan.

Furthermore, it also includes: a data verification unit, which is used to verify the data after compliance verification and generate a verification result table for operation and maintenance personnel to review the verification results and urge the power plant to rectify the problems.

In a third aspect, the present invention provides a network source data compliance verification system, comprising:

The network source data import and export module is used to provide an interface for users to import network source related data into E-text format files according to the database table or import the corresponding E-text files into the system;

Intelligent verification module, used to configure data verification rules. For similar functional parameters, it provides one-click configuration of rule templates and combines rule search algorithms to perform intelligent verification of network source data.

The non-compliant information alarm module is used to display corresponding alarm information for non-compliant data;

The rectification plan push module is used to input non-compliant information into the rectification plan model, generate rectification plans based on the rectification plan model and push them to the corresponding system specialists to urge system rectification;

The rectification plan self-learning module is used to provide historical rectification plans and put them into the model for reinforcement learning training so that the rectification plan model can be continuously improved.

In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the aforementioned methods.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention provides a network source data compliance verification method, device and system, which can quickly locate missing data and non-compliant data, provide new rectification plans for customers to choose and use, and can well discover and solve various data compliance problems in network source coordinated operation and maintenance.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a method flow chart of a network source data compliance verification method provided by an embodiment of the present invention;

FIG. 2 is a schematic diagram of the structure of a network source coordination system provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

Example 1

This embodiment introduces a network source data compliance verification method, including:

Obtain relevant documents pre-entered or uploaded from the power plant, perform OCR text image recognition on the relevant documents, extract network source data information and store it in the database;

Extracting network source data information that needs to be verified from the database;

Performing verification rule configuration on the network source data information;

Perform compliance verification according to the verification rule information to filter out non-compliant data;

The non-compliant data is input into a pre-built rectification plan model to generate a new rectification plan.

The network source data compliance verification method provided in this embodiment specifically involves the following steps:

Step 1: Data extraction. Extract the network source data information that needs to be verified from the MySQL or DM database.

Step 2: Configure validation rule information. First, configure different validation rules for different table data, including the qualified range of different data and the validity period of the file.

The business rule dynamic configuration management method provided includes:

1. Based on the rule engine framework - Drools, establish a network source business rule library and configure corresponding business rules according to different business categories;

2. Obtain verification request information, and use the RETE algorithm to match rules in the rule engine according to the request information to obtain one or more verification logic rules. When multiple logic rules are obtained, prioritize the multiple logic rules and select the optimal verification logic rule;

3. According to the optimal business logic rules, the corresponding service is called and executed.

Step 3: Perform compliance verification according to the verification rule information. Perform multi-threaded verification processing according to the number of extracted data tables.

Step 4: Data verification. After the verification is completed, a verification result table is generated for the operation and maintenance personnel to review the verification results and urge the power plant to rectify the problems.

Step 5: Generate a rectification plan. For each non-compliant data, the correct interval value is generated through big data analysis combined with information such as device model and device parameters to provide to the user, so that the user can make rectifications;

Step 6: The warning reason will be synthesized into a voice message through intelligent voice. And the contact person of the warning plant station will be automatically called by phone, so as to realize automatic warning push;

Convolutional neural network and supervised learning were used to learn the rectification plan and obtain the optimal rectification plan.

The contents designed in the above embodiment are described below in conjunction with a preferred embodiment.

(1) The power plant uploaded five test reports and filled in network source data such as the steam turbine parameter table and gas generator set information table;

(2) The system automatically extracts the network source compliance information contained in the test documents and stores it in the database;

(3) Perform compliance verification on the power plant data at the specified time according to the verification cycle set in the program;

(4) The verification results showed that the power plant was missing one AVC test report, and the data in the turbine main steam pressure field of the turbine parameter table was abnormal, and the possible range was (12--14MPa);

(5) The power plant re-uploads the missing files and modifies the corresponding data tables as required;

(6) Intelligently generate rectification plans based on the rectification plan model.

A. Using a deep learning method based on a convolutional neural network (CNN), a top-down supervised learning approach is used to train the rectification plan for previous data non-compliance to obtain a rectification plan model.

B. When there are non-compliant items in the reported network source data, it is sent to the rectification plan model obtained through the above training. The rectification plan model can intelligently synthesize new rectification plans from historical rectification plans based on the non-compliant items.

C. For each non-compliant data or several non-compliant data, different rectification plans are generated by comparing and analyzing with the historical data of the item. Among the generated rectification plans, those that are conflicting or have similar effects can be selected.

D. In order to improve the accuracy of the rectification plan, when generating the rectification plan, multiple rectification plans are returned by sorting by the degree of matching, and the user can choose a more suitable rectification plan. After the user selects the appropriate rectification plan, the selected plan will be put into the model for reinforcement learning training to continuously improve the rectification plan model.

Example 2

This embodiment provides a network source data compliance verification device, including:

A network source data information storage unit is used to obtain relevant files pre-entered or uploaded from the power plant, perform OCR text image recognition on the relevant files, extract network source data information and store it in a database;

A network source data information acquisition unit, used to extract network source data information that needs to be verified from the database;

A verification rule configuration unit, used to configure verification rules for the network source data information;

A compliance verification unit, used to perform compliance verification according to the verification rule information and filter out non-compliant data;

The rectification plan generating unit is used to input the non-compliant data into a pre-built rectification plan model to generate a new rectification plan.

Furthermore, it also includes: a data verification unit, which is used to verify the data after compliance verification and generate a verification result table for operation and maintenance personnel to review the verification results and urge the power plant to rectify the problems.

Example 3

This embodiment provides a network source data compliance verification system, including:

The network source data import and export module is used to provide an interface for users to import network source related data into E-text format files according to the database table or import the corresponding E-text files into the system;

Intelligent verification module, used to configure data verification rules. For similar functional parameters, it provides one-click configuration of rule templates and combines rule search algorithms to perform intelligent verification of network source data.

The non-compliant information alarm module is used to display corresponding alarm information for non-compliant data;

The rectification plan push module is used to input non-compliant information into the rectification plan model, and automatically send text messages or make phone calls to the corresponding system specialists through speech synthesis audio based on the rectification plan generated by the rectification plan model to urge system rectification;

The rectification plan self-learning module is used to provide historical rectification plans and put them into the model for reinforcement learning training so that the rectification plan model can be continuously improved.

Example 4

This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any method described in Embodiment 1 are implemented.

The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims (8)

1. The network source data compliance verification method is characterized by comprising the following steps of:

acquiring related files which are input or uploaded from a power plant in advance, performing OCR text image recognition on the related files, extracting network source data information and storing the network source data information into a database;

extracting network source data information to be checked from the database;

Performing verification rule configuration on the network source data information;

carrying out compliance verification according to the verification rule information, and screening out non-compliance data;

inputting the non-compliance data into a pre-constructed modification scheme model to generate a new modification scheme;

the configuring the verification rule for the network source data information comprises the following steps:

different verification rules are configured for different data tables in the network source data information to form a verification rule base, wherein the verification rule base comprises one or more of seal or signature missing, qualified range of different data and validity period of a file;

the network source data information is subjected to verification rule configuration, and a service rule dynamic configuration management method is provided, which comprises the following steps:

based on rule engine frame-Drools, establishing network source service rule base, and configuring corresponding service rules according to different service categories;

Acquiring verification request information, carrying out rule matching in a rule engine by adopting a RETE algorithm according to the request information, acquiring one or more verification logic rules, when acquiring a plurality of logic rules, carrying out priority ordering on the plurality of logic rules, and selecting an optimal verification logic rule;

and calling the corresponding service and executing the service through the optimal business logic rule.

2. The network source data compliance verification method according to claim 1, wherein when the compliance verification is performed according to the verification rule information, the multi-thread verification processing is performed according to different data tables.

3. The network source data compliance verification method of claim 1, further comprising: and (3) carrying out data verification on the data subjected to the verification, generating a verification result table for operation and maintenance personnel to review the verification result and prompting the power plant to carry out problem correction.

4. The network source data compliance verification method according to claim 1, wherein the constructing of the rectification scheme model includes:

and (3) using a deep learning method based on a convolutional neural network, and performing model training on the conventional correction scheme with the data non-compliance in a top-down supervised learning mode to obtain a correction scheme model.

5. The network source data compliance verification method of claim 1, further comprising: when generating the modification schemes, a plurality of modification schemes are returned for selection by a user according to the matching degree sequence.

6. A network source data compliance verification apparatus employing the network source data compliance verification method of claim 1, comprising:

The network source data information storage unit is used for acquiring related files which are input or uploaded from the power plant in advance, performing OCR text image recognition on the related files, extracting network source data information and storing the network source data information into the database;

The network source data information acquisition unit is used for extracting network source data information to be checked from the database;

the verification rule configuration unit is used for carrying out verification rule configuration on the network source data information;

The compliance verification unit is used for carrying out compliance verification according to the verification rule information and screening out non-compliance data;

and the rectification scheme generating unit is used for inputting the non-compliance data into a pre-constructed rectification scheme model to generate a new rectification scheme.

7. A network source data compliance verification system employing the network source data compliance verification method of claim 1, comprising:

The network source data importing and exporting module is used for providing an interface for a user to import network source related data into E text format files or import corresponding E text files into the system according to the database table;

the intelligent verification module is used for configuring data verification rules, providing rule template one-key configuration for similar functional parameters, and carrying out intelligent verification on network source data by combining a rule searching algorithm;

The non-compliance information alarm module is used for displaying corresponding alarm information for the non-compliance data;

the correction scheme pushing module is used for inputting the non-compliance information into the correction scheme model, pushing the correction scheme generated based on the correction scheme model to the corresponding system responsibility, and prompting the system correction;

and the correction scheme self-learning module is used for providing a historical correction scheme put-in model for reinforcement learning training so that the correction scheme model is continuously improved.

8. A computer-readable storage medium having stored thereon a computer program, characterized by: the program, when executed by a processor, implements the steps of the method of any of claims 1 to 5.